{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "0b9ffbb2-8d33-eb0c-5a42-0733bdc3b74b",
    "_uuid": "e14f65723f5ac826104e861c45e58525740d8b2a"
   },
   "source": [
    "# Bike Sharing 数据集上的回归分析\n",
    "\n",
    "1、 任务描述 请在Capital Bikeshare （美国Washington, D.C.的一个共享单车公司）提供的自行车数据上进行回归分析。训练数据为2011年的数据，要求预测2012年每天的单车共享数量。\n",
    "\n",
    "原始数据集地址：http://archive.ics.uci.edu/ml/datasets/Bike+Sharing+Dataset 1) 文件说明 \n",
    "day.csv: 按天计的单车共享次数（作业只需使用该文件） \n",
    "hour.csv: 按小时计的单车共享次数（无需理会） \n",
    "readme：数据说明文件\n",
    "\n",
    "2) 字段说明 \n",
    "Instant记录号 \n",
    "Dteday：日期 \n",
    "Season：季节（1=春天、2=夏天、3=秋天、4=冬天） \n",
    "yr：年份，(0: 2011, 1:2012) \n",
    "mnth：月份( 1 to 12) \n",
    "hr：小时 (0 to 23) （只在hour.csv有，作业忽略此字段） \n",
    "holiday：是否是节假日 \n",
    "weekday：星期中的哪天，取值为0～6 \n",
    "workingday：是否工作日 1=工作日 （是否为工作日，1为工作日，0为非周末或节假日 weathersit：天气（1：晴天，多云 ",
    "2：雾天，阴天 ",
    "3：小雪，小雨 ",
    "4：大雨，大雪，大雾） temp：气温摄氏度 \n",
    "atemp：体感温度 \n",
    "hum：湿度 \n",
    "windspeed：风速 \n",
    "\n",
    "casual：非注册用户个数 \n",
    "registered：注册用户个数 \n",
    "cnt：给定日期（天）时间（每小时）总租车人数，响应变量y\n",
    "casual、registered和cnt三个特征均为要预测的y，作业里只需对cnt进行预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_cell_guid": "2ba3154a-c2aa-3158-1984-63ad2c0c786a",
    "_uuid": "5eb696b95780825e94ddb49787f9fa339fc3833b"
   },
   "outputs": [],
   "source": [
    "# 导入必要的工具包\n",
    "# 数据读取及基本处理\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "#模型\n",
    "from sklearn.linear_model import LinearRegression, RidgeCV, LassoCV, ElasticNetCV\n",
    "\n",
    "#模型评估\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.metrics import r2_score  #评价回归预测模型的性能\n",
    "\n",
    "#可视化\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_cell_guid": "21fa35be-878b-b4f2-ef6e-68dc070b8bfa",
    "_uuid": "73aee228226be55c0c8a6e4fcbf818c56cd94926"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>yr</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.355170</td>\n",
       "      <td>0.373517</td>\n",
       "      <td>0.828620</td>\n",
       "      <td>0.284606</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.379232</td>\n",
       "      <td>0.360541</td>\n",
       "      <td>0.715771</td>\n",
       "      <td>0.466215</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.144830</td>\n",
       "      <td>0.449638</td>\n",
       "      <td>0.465740</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.175530</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.607131</td>\n",
       "      <td>0.284297</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.209120</td>\n",
       "      <td>0.197158</td>\n",
       "      <td>0.449313</td>\n",
       "      <td>0.339143</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   instant  season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  \\\n",
       "0        1         1         0         0         0       1       0       0   \n",
       "1        2         1         0         0         0       1       0       0   \n",
       "2        3         1         0         0         0       1       0       0   \n",
       "3        4         1         0         0         0       1       0       0   \n",
       "4        5         1         0         0         0       1       0       0   \n",
       "\n",
       "   mnth_4  mnth_5  ...   weekday_5  weekday_6      temp     atemp       hum  \\\n",
       "0       0       0  ...           0          1  0.355170  0.373517  0.828620   \n",
       "1       0       0  ...           0          0  0.379232  0.360541  0.715771   \n",
       "2       0       0  ...           0          0  0.171000  0.144830  0.449638   \n",
       "3       0       0  ...           0          0  0.175530  0.174649  0.607131   \n",
       "4       0       0  ...           0          0  0.209120  0.197158  0.449313   \n",
       "\n",
       "   windspeed  holiday  workingday  yr   cnt  \n",
       "0   0.284606        0           0   0   985  \n",
       "1   0.466215        0           0   0   801  \n",
       "2   0.465740        0           1   0  1349  \n",
       "3   0.284297        0           1   0  1562  \n",
       "4   0.339143        0           1   0  1600  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 读入数据\n",
    "data = pd.read_csv(\"FE_day.csv\") #读入经过数值特征处理之后的文件\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "a1ae46d1-8787-67a3-2f69-6497c97320eb",
    "_uuid": "16fa6600f9877c9607060f9e50de6eaa9bc1c766"
   },
   "source": [
    "**准备训练数据**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# get labels\n",
    "y = data['cnt']   \n",
    "X = data.drop(['cnt'], axis=1)#去掉标签数据，得到样本特征数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train samples: (584, 34)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\dell\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "# 用train_test_split 分割训练数据和测试数据\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, train_size = 0.8,random_state = 0) #完成样本训练数据集和测试数据集的分割\n",
    "\n",
    "print(\"train samples:\" ,X_train.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\dell\\Anaconda3\\lib\\site-packages\\pandas\\core\\frame.py:3694: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  errors=errors)\n"
     ]
    }
   ],
   "source": [
    "#保存测试ID，用于结果提交\n",
    "testID = X_test['instant']\n",
    "\n",
    "#ID不参与预测\n",
    "X_train.drop(['instant'], axis=1, inplace = True)\n",
    "X_test.drop(['instant'], axis=1, inplace = True)\n",
    "\n",
    "#保存特征名字以备后用（可视化）\n",
    "feat_names = X_train.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "9fe1c63e-3803-feac-362c-631399fdb8ec",
    "_uuid": "431749cca6d8fbbc2d1d3732baeb6d63590c6e2e"
   },
   "source": [
    "**1* Linear Regression without regularization**\n",
    "最小二乘线性回归\n",
    "最小二乘没有超参数需要调优，直接用全体训练数据训练模型，没有加正则项"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "_cell_guid": "101cf15a-9006-ac9a-8abe-756371fd8b1a",
    "_uuid": "38562579d6306877a067189f25ce15ececbe99d2",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE on Training set : 752.7789793696566\n",
      "RMSE on Test set : 782.8217275618103\n",
      "r2_score on Training set : 0.8434599824797839\n",
      "r2_score on Test set : 0.8559159938127985\n"
     ]
    }
   ],
   "source": [
    "# Linear Regression\n",
    "# 1. 生成学习器实例\n",
    "lr = LinearRegression()\n",
    "\n",
    "#2. 在训练集上训练学习器\n",
    "lr.fit(X_train, y_train)\n",
    "\n",
    "#3.用训练好的学习器对训练集/测试集进行预测\n",
    "y_train_pred = lr.predict(X_train)\n",
    "y_test_pred = lr.predict(X_test)\n",
    "\n",
    "\n",
    "rmse_train = np.sqrt(mean_squared_error(y_train,y_train_pred))\n",
    "rmse_test = np.sqrt(mean_squared_error(y_test,y_test_pred))\n",
    "print(\"RMSE on Training set :\", rmse_train)\n",
    "print(\"RMSE on Test set :\", rmse_test)\n",
    "\n",
    "r2_score_train = r2_score(y_train,y_train_pred)\n",
    "r2_score_test = r2_score(y_test,y_test_pred)\n",
    "print(\"r2_score on Training set :\", r2_score_train)\n",
    "print(\"r2_score on Test set :\", r2_score_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(33,)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr.coef_.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ridge picked 33 features and eliminated the other 0 features\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAacAAAEVCAYAAABE7SrmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XvcpXO9//HX21nOZqZyGhNDB9LIOBaGpJJDihSVqWSXbL8SsmOXlP1LU6hEITaSbIqGSKLJoRkZjJlBGOfZ5DAhk0MOn/3H93vPLGvW+V6Ha615Px+P9bjX4Tp81n1Yn/u6ru/1vhQRmJmZFckSvS7AzMysnJuTmZkVjpuTmZkVjpuTmZkVjpuTmZkVjpuTmZkVjpuTDSRJb5Z0q6RnJR0iaXlJl0p6RtKFkvaT9PsGlvM1SWd0o+YaNYyWNF/Skm1a3jGSft6OZQ2CZr4fkqZIOqDTNZmbk/WYpH0lTc8fvo9KukLSu9uw6COAKRGxUkT8ENgLeAMwIiL2jojzImLneguJiP+KiGF/GEkaIykkLdXsvBHxUESsGBGvtLDeCZLmNjtfk+vYRtI1+R+BZ/I/AW9rpAZJa0v6laQn87yzJE2sMu2E/D38ddnz78jPT2nn+7LecnOynpF0KHAS8F+kxjEaOAXYow2LXxe4vezx3RHxchuWbZmkrYHfA78B1gTeBNwG3CBpvQYWcS7wMOnnMwL4FPBYjemfALaRNKLkuf2Bu5uv3gotInzzres3YBVgPrB3jWmWJTWvR/LtJGDZktd3BWYATwN/BjbJz18DvAK8kNdxPvAv4KX8+LPAROD6kmVtBFwF/J304fi1/PwxwM9Lptsqr+tp0ofwhJLXpgDfAm4AniV9aI/Mrz0ERF7/fGBrYCzwJ+AZ4EnggirfhzF53qXqradsvhWA54FXS9a7Zn5P/wOck+e/HRhfMt+awK9IjeB+4JAaP6PrgFMqPH8FcE6+PwGYW2X++cC4Bn9nJgBzgZ8AX8zPLZmf+zppS3lo2m2Am/L39iZgm5LX3pS/78/mn/nJTf6MD+j138/icOt5Ab4tnjfg/cDLQx+4VaY5FpgGvB4YlT8wvpVfeyfwOLBl/oDaH3iA3LzKP0RYtMlMJDcnYCXgUeArwHL58Zbl8wFrAfOAXUh7Hd6bH48qWee9wIbA8vnxd/JrYyhpMPm584Gj8rKWA95d5fvwmnlrrafCvIs0hvyeXsjvY0ng/wPT8mtLADfnD/tlgPWA+4D3VVj260j/BOxQ4bVPA49Wq6Fkuj+QmuzHgNF1fmcmkBrRNsCN+bldgCuBA8jNCVgdeAr4JLAU8PH8eER+fSpwAumfn+1ITaqZn7GbUxdu3q1nvTICeDJq72bbDzg2Ih6PiCeAb5I+cAA+B/w0Im6MiFci4mzgRdJ/vc3aFfhbRHw/Il6IiGcj4sYK030CuDwiLo+IVyPiKmA66YNsyFkRcXdEPE/aOhlXY70vkXZnrZnXe30TNTeznkquz+/jFdKutXfk5zcnfRAfGxH/ioj7gNNJzaPc6qQP8EcrvPYoMLKBOvYmbX39J3C/pBmSNq81Q0T8GVhd0ptJuwHPKZvkg8A9EXFuRLwcEecDfwV2kzQ6v8f/jIgXI+Ja4NKSeRv5GVsXuDlZr8wDRtYZILAm8GDJ4wfzc5A+1L8i6emhG7BOyevNWIe0JVLPusDeZet8N7BGyTR/K7n/HLBijeUdAQj4i6TbJX2miZqbWU8j8y+XfxbrAmuWvcevkY4JlnuKtMtwjQqvrUHaVVlTRDwVEUdGxEZ5HTOASySpzqznAgcDOwAXl71W/ntDfrxWfu2piPhn2WtDGvkZWxc0PXLIrE2mknYtfQi4qMo0j/DagQ2j83OQDqIfFxHHtaGWh0m7fhqZ7tyI+FwL61gk/j8i/kbaAiSPUPyDpGsjYk4Ly294vXU8DNwfERvUXXDEPyVNJW39/LHs5Y8CVzez4oh4UtL3SLtoVyf9A1PNucAc0nGt58p62dDvTanRwO9IW3SrSVqhpEGNZuH3aTg/Y2sjbzlZT0TEM6TjGj+W9CFJr5O0tKQPSPpunux84GhJoySNzNMPnY9yOvB5SVsqWUHSByWt1EI5lwFvlPQlSctKWknSlhWm+zlp19D7JC0pabk8vHntBtbxBGkrY8EINkl7l8z7FOkDsunh4nU8BoyQtEqD0/8F+Iekr+Zzw5aUtHGNXW1HAvvnc8lWkrSapG+TBnx8s3TC/P0qvUnS8Xn5S+Wf3ReAORFRqzEREfcD25OO2ZW7HNgwn6awlKR9gLcBl0XEg6TddN+UtEz+p2C3knmH8zO2NnJzsp6JiBOAQ4GjSR/eD5N21VySJ/k26YNkJjALuCU/R0RMJ211nEz6YJ9DGuTQSh3Pkg5870ba3XUPaXdR+XQPk4a5f62k3sNp4O8oIp4DjiMNsX5a0lakYx83SpoPTAb+X/7QbZuI+Cupyd+X11tzt2c+BrUb6RjW/aRdc2eQRldWmv564H3Ah0lbJQ8Cm5IGd9xTMulapJGDpbf1SYMqLiaNjLuPtMWze4Pv7fqIeKTC8/NIxxG/Qtr6OgLYNSKGdjPuSxpI83fgG5QcsxrOz9jaSxG+2KCZmRWL/xswM7PCcXMyM7PCcXMyM7PCcXMyM7PC8XlOLRo5cmSMGTOm12WYmfWVm2+++cmIGFVvOjenFo0ZM4bp06f3ugwzs74iqTy9oyLv1jMzs8LxlpOZDZQxR/621yUMvAe+88GOr2NgtpwkrSrpoJLHEyRd1sT8O0q6RdJsSWfXCSQ1M7MOGpjmBKwKHFR3qgokLQGcDXwsIjYmRbDs38bazMysCYVqTpLGSPqrpDPyFsx5knaSdIOkeyRtIekYSWdKmiLpPkmH5Nm/A6yfrwczKT+3oqSL8jLPqxHDPwJ4MSKGLvV8FfCRCvUdKGm6pOlPPPFEW9+7mZktVKjmlI0FfgBsAryFFNL4buAwUhgj+fn3AVsA35C0NCkd+d6IGBcRh+fpNgW+REokXg94V5V1PgksLWl8frwX6Ro/rxERp0XE+IgYP2pU3ZGQZmbWoiI2p/sjYlZEvEq6js/VkdJpZ5EuVw3w23wVyydJl+qudCE0gL9ExNy8rBkl879GXv7HgBMl/YV02eZaV2g1M7MOKuJB/xdL7r9a8vhVFtZbOs0rVH8fjU5HREwFtgWQtDOwYeMlm1lRdGMkmXVeEbecWvUs0MqF5gCQ9Pr8dVngq8BP2lSXmZk1aWCaU77A2A15IMWkujMs6nBJd5IubHdpRFzT3grNzKxRvthgi8aPHx+OLzIza46kmyNifL3pBmbLyczMBkcRB0S0RNKqwL4RcUp+PAE4LCJ2LZvuYuBNZbN/Ffg4sD3wTH5uYkTM6GjRZtZ2ji/qvG4MOhmY5sTChIhTak0UEXtWel7Sx4HDI+KiDtRmZmZNKNRuvR4mRJiZWYEUqjllvUiIGHKcpJmSTsxDyl/D8UVmZt1RxObU9YSI7D9ITW9zYHXScajXcHyRmVl3FPGYU68SIh4dmkfSWaQtNTPrM06IGAxF3HJq1XATItbIXwV8CJjdprrMzKxJRdxyaklEzMsDJ2YDVwDNjic9T9IoQKRdgJ9vd41mZtYYJ0S0yAkRZmbNc0KEmZn1rYHZrdeoagkREXFlL+oxM7NFDUxzajS+qEZCxM+A8aRjTneT4ovmd7RoM2s7xxd1XjdGRA7Sbr2h+KJWfTki3hERmwAPAQe3pywzM2tWoZpTL+OLIuIfuQYBywMeKWJm1iOFak5Zz+KL8sm3f8vL/1GF1x1fZGbWBUVsTr2KLyIiPg2sCdwJ7FPhdccXmZl1QREHRPQkvmhIRLwi6QLgcOCsRgo2s+JwfNFgKOKWU6taji9SMnboPrAb8Nc21mZmZk0o4pZTS4YZXyTgbEkr5/u3AV/oQJlmZtYAxxe1yPFFZmbNc3yRmZn1rYHZrdcoxxeZmRVfXzWnRiOKasx/MPB2YH1gVB6KPjQg4ofALsBzpOiiWzrwFsyswxxf1HmOL1rUcCOKbgB2Ah4se/4DwAb5diBw6jDWYWZmw9T15tTjiKJbI+KBCi/tAZwTyTRg1aEr45bV7oQIM7Mu6NWWU88iiqpYC3i45PHc/NxrOCHCzKw7etWcehZRVEWlrS2PsTcz65FeDYjoaURRBXOBdUoerw080sJyzKzHHF80GPptQETLEUV1TAY+lUftbQU8ExGPdmA9ZmbWgL5qThExD7ghD6SYVHeGMpIOkTSXtGU0U9IZ+aXLgfuAOcDpDG9EoJmZDZPji1rk+CIzs+Y5vsjMzPpWXyVENMoRRWZm/W1gmlNptFFE7NlCtNF7gEmkrcn5pAijOR0r2Mw6wvFFnef4ouYMN9roVGC/iBgH/AI4ui1VmZlZ0wrVnHoZbUQ66XblfH8VKpzn5PgiM7PuKOJuvbHA3qQA1ptYGG20OynaaAYp2mgH0jlPd0k6lRRttHHe8hlKLN8U2IjUaG4gRRtdX2W9BwCXS3oe+AewVfkEEXEacBqk0XrDf6tmZlZJobacsl5FG30Z2CUi1gbOAk4Y/lsxM7NWFHHLqevRRpJGAe+IiBvzUxcAv2uiZjMrCMcXDYYibjm1ajjRRk8Bq0jaMD9+L3BnW6oyM7OmFXHLqSURMS8PnJgNXAE0PJ40Il6W9DngV5JeJTWrz3SoVDMzq8PxRS1yfJGZWfMcX2RmZn1rYHbrNcrRRmaDzQkRneeEiCZIWlXSQSWPJ0i6rHy6iNgzX+a99HZlyXw/kjS/W3WbmdmiBqY5Mfz4IiSNz8sxM7MeKlRz6mV8kaQlScGvR9SYxvFFZmZdUKjmlI0FfgBsQoopGoovOowUX0R+/n3AFsA3JC1Nii+6N++mOzxPtynwJeBtwHqk+KJqDgYm17o8e0ScFhHjI2L8qFGjWn1/ZmZWRxGbU9fjiyStScrz+1H73oaZmbWqiKP1uh5fRNrCGgvMyXv+XidpTkSMbaJuMysAxxcNhiI2p1a1HF8UEb8F3jj0WNJ8NyYzs94p4m69lkTEPOCGPJBiUt0ZzMyssBxf1CLHF5mZNc/xRWZm1rcG6ZhTQxxfZDbYHF/UeY4vasJw44uUHCfpbkl3lpzca2ZmXTYwzYnhxxdNBNYB3hIRbwV+2Y6izMyseYVqTr2MLwK+ABybT9glIh6vUJ/ji8zMuqBQzSnrVXzR+sA+uflcIWmD8gkcX2Rm1h1FbE5djy/KlgVeyEMcTwfOHP5bMTOzVhRxtF4v4osA5gK/yvcvBs5qpFgzKxbHFw2GIm45tarl+KLsEmDHfH974O5hV2RmZi0p4pZTSyJiXh44MRu4Amj2ZIfvAOdJ+jIwHzig3TWamVljHF/UIscXmZk1z/FFZmbWtwZmt16jHF9kNtgcX9R53Rh0MjDNSdKqwL4RcUp+PAE4LCJ2LZ0uIvasMv95wHjgJeAvwL9FxEsdLdrMzCoapN16w40vOo90cu/bgeXxgAgzs54pVHPqZXxRRFweGWnLae0K9Tm+yMysCwrVnLJexRcBkJf1SeB35a85vsjMrDuK2Jx6FV805BTg2oi4bjhvwszMWlfEARG9ii9C0jeAUcC/NVqsmRWL44sGQxGbU6uGFV8k6QDSrsL3DF02w8zMeqOIu/VaEhHzgBvyQIpJdWdY1E9Iuwen5kEVX29vhWZm1ijHF7XI8UVmZs1zfJGZmfWtQTrm1BDHF5kNNscXdV43Bp0MzJaTpFUlHVTyeIKky8qni4g987lQpbcrJR0saY6kkDSyu9WbmVmpgWlODD++6AZgJ+DB9pRjZmatKlRz6nF80a0R8UCd+hxfZGbWBYVqTllP44tqcXyRmVl3FHFAxP0RMQtA0oL4IklD8UUzyPFFwIuS6sYX5WUNxRdd3+H6zayHnBAxGIq45dSz+CIzMyuGIjanVg0rvsjMzIpjYJrTcOOLJB0iaS7pOk4zJZ3R9iLNzKwhji9qkeOLzMya5/giMzPrW301QEDSqsC+EXFKfjwBOCwidm1w/oNJ50OtAMwmDZKAdNXbXfL9+cAXIuK2NpZuZl3i+KLOc3zRotqRArExKQVih6H4ImAysH1EbAJ8Czht2JWamVnLut6cipgCERF/join8sNppEERZmbWI73arTcW2Bs4ELiJhSkQu5NSIGaQUiB2IA0Pv0vSqaQUiI3z1s7Qbr1NgY2AR0hbRu9ieCfafha4otILkg7MNTN69OhhrMLMzGrp1W69+yNiVr4c+oIUCGAoBQJyCkREPAnUTYHIy5pRMn/TJO1Aak5frfS644vMzLqjV1tOhUuBkLQJcAbwgXzOlJn1IccXDYZ+GxDRkRQISaOBXwOfjIi72718MzNrTl81pw6mQHwdGAGckgdb+OxaM7MeckJEi5wQYWbWPCdEmJlZ3+qrhIhGSboYeFPZ01+NiCt7UY+ZmTWncM1J0hRSJFHVfWaSJgLjI+LgSq9HxJ5NrnMiMAn43/zUyRHhVHLra4trjI9H6w2GwjWnHrqgWrMzM7PuGvYxJ0lHDMULSTpR0jX5/nsk/VzSzpKmSrpF0oWSVsyvbybpT5JulnSlpDXKlruEpLMlfTs//rSkuyX9iZQCMTTdbpJulHSrpD9IekOe9x5Jo0qWNUfSyOG+XzMz67x2DIi4Ftg23x9PyrpbmhRHNAs4GtgpIt4JTAcOza//CNgrIjYDzgSOK1nmUsB5wN0RcXRuXN8kNaX3Am8rmfZ6YKuI2BT4JXBETov4ObBfnmYn4LacNlHNRyTNzDl961SaQNKBkqZLmv7EE0808K0xM7NWtKM53QxsJmklUlrDVFKT2hZ4ntRIbpA0A9gfWBd4Mykd/Kr8/NG8Nmz1p8DsiBhqWFsCUyLiiYj4F3BBybRrA1dKmgUcTsrZg9TwPpXvfwY4q8Z7uBQYk1PJ/wCcXWkixxeZmXXHsI85RcRLkh4APg38GZhJCmxdH7gfuCoiPl46j6S3A7dHxNZVFvtnYAdJ34+IF4ZWVWXaHwEnRMTkHAR7TK7rYUmPSdqR1Nz2qzI/ZXFFpwPHV5vWrF94YID1s3ad53QtcFj+eh3weVII6zTgXZLGAkh6naQNgbuAUZK2zs8vLWmjkuX9DLgcuFDSUsCNwARJI/Iuwb1Lpl2FhaPs9i+r6wzS7r3/iYhXqKLseNfuwJ0Nv3MzM2u7djWn64A1gKkR8RjwAnBdRDwBTATOlzST1KzeknfN7QUcL+k2UiPbpnSBEXECcAtwLvAYaYtoKmm32y0lkx5DamLXAeXHlCYDK1J7lx7AIZJuz7Uckms2M7MeGej4IknjgRMjYtu6EzfJ8UVmZs1rNL5oYM9zknQk8AVqHGsyM7NiGtjmFBHfIV3WfQFJR/Ha41UAF5aMCjQzswIoXHNqR3xRNbkJVW1EkvYCLgQ2r7V+s37g+CLrZ04lz/J5WoeQRgaamVkPOb5ooW8B3yWNNKz2Xp0QYWbWBY4vAiRtCqwTEZfVeqNOiDAz6452HHMqjy+6hYXxRZNZGF8EsAzpXKXS+CKAJYFHS5b5U9KJs4vEFwFIugDYML+2NnBBbmDLkFIpIDW83wAnUSO+SNISwIn43CYzs8JwfBGsRGqUU3KjfCMwWdLuHhRh/cwDA6yfLfbxRRHxTESMjIgxETEm1+zGZGbWQ44vMjOzwnF8UYscX2Rm1jzHFzm+yMysbw1sc3J8kZlZ/ypcc+pFfJGkzwNfBF4B5gMHRsQdzSzbrGgcX2T9zPFFyS8i4u0RMY6UEnFCrwsyM1ucOb4IiIh/lDxcgSrnVDm+yMysOxxflEn6oqR7SVtOh1SaxvFFZmbdsdjHFw2JiB8DP5a0L6mhlp/Qa2ZmXeL4okX9Eji1wWnNCssDA6yfLfbxRXn9G5Q8/CBwT8Pv3MzM2s7xRcnBkm6XNAM4FO/SMzPrKccXtcjxRWZmzXN8keOLzMz61sA2J8cX2eLOCRHWzwqXECFpSt4dV2uaiZJObnbZEXFcRIwrux0n6VBJd0iaKelqSeu2/g7MzGy4CteceuRWUlbfJsBFpBNxzcysRxxfBETEHyPiufxwGunE3krv1fFFZmZd4PiiRX0WuKLSC44vMjPrDscXlZD0iVz79k19B8zMrK0cX7Swpp2Ao4DtI+LFWtOa9QOPWrN+5viitP5NSVtru0fE402/ezMzayvHFyWT8nQXSpohaXLD79zMzNrO8UUtcnyRmVnzHF/k+CIzs741sM3J8UW2uHN8kfWzwiVE9Ci+aLt8kvDLkvZqvXozM2uHwjWnHnmINHDjFz2uw8zMcHwRABHxQETMBF6t814dX2Rm1gWOL2qC44vMzLrD8UVmZlY4ji8yG1AetWb9zPFFZmZWOI4vAiRtLmkuqen9VNLtzbx5MzNrL8cXtcjxRWZmzXN8keOLzMz61sA2J8cXWVEtrrFC3eKBIIOhcAkRvYgvysv8qKQ7JN0uyUkRZmY9NLBbTs2QtAHwH8C7IuIpSa/vdU1mZoszxxclnwN+HBFPAVS7Gq7ji8zMusPxRcmGwIaSbpA0TdL7K03k+CIzs+5wfFGyFLABMCEv7zpJG0fE001+L8zMrA0cX5TMBaZFxEvA/ZLuIjWrm2rMY9YSjyYzq8/xRcklpIZKPi61IXBfE+/fzMzayPFFyZXAPEl3AH8EDo+IeY2+eTMzay/HF7XI8UVmZs1zfJHji8zM+lbhmpOkKcBhEVF1s0TSRGB8RBxcbZpm4ouA7wHnAJsB84B9IuKBFso3q8vxRZ3lASeDoXDNqZPy0PRFcvQkHQQ8FRFjJX0MOB7Yp9v1mZlZ4oSIZA/g7Hz/IuA9yidgmZlZ9zkhIlkLeBggIl4GngFGlE/k+CIzs+5wQkRSaStpkWGMEXEacBqk0Xo1lmdmZsPghIhkLrAOMDef9LsK8Pca05u1zAfszepzQkQyuWTevYBrYpBPADMzKzgnRCQ/A0ZImgMcChzZ6Bs3M7P2c0JEi5wQYWbWPCdEOCHCzKxvDWxzaiYhomRUoJmZFUDXm1OOHvp9RDySHz9AiiKqdg5Sq+u5HNg3P9w3Ik6plhCRp/8dsBVwfUTs2s5aKnGEjVlneDTkYGjXgIhmTATWbMeC8ki+iiJil3wl21WBgxpY3CTgk+2oy8zMhqducxpGPNHXJd0kabak05TsRTpB9zxJMyQtn1fz73n+WZLekudfQdKZeRm3StojPz8xr+dS4PeS1pB0bV7ebEnb5ukeyHFF3wHWz69PqvY+I+Jq4NkWv49mZtZGjWw5NR1PlKc9OSI2j4iNgeWBXSPiojzNfhExLiKez9M+mec/lXS+FMBRpPONNied1DtJ0gr5ta2B/SNiR9KuuysjYhzwDtKw9FJHAvfm9R3eyDelGscXmZl1RyPNqTyeaCoL44meZ2E80QzSiazr5vl2yIGss4AdgY0WWfJCvy5Z15h8f2fgyLzcKcBywOj82lURMZTgcBPwaUnHAG+PiI5t/UTEaRExPiLGjxo1qlOrMTNb7NUdENFiPNFywCmkgQ4P58axXI3VvJi/vlJSk4CPRMRdZcveEvhnSX3XStoO+CBwrqRJEXFOvffVaz5oa2ZWXaMDIpqNJxpqRE/mY1B7lSzrWWClBtZ5JelYlPKyN600kaR1gccj4nRS0sM7yyZpdH1mZlYQjTanZuOJngZOJx2TuoS0623IfwM/KRsQUcm3gKWBmZJm58eVTABmSLoV+Ajwg9IXI2Ieabfj7FoDInL80YWkaznNlfS+GrWZmVkHDXR8USc5vsjMrHmNxhf14jwnMzOzmgY2vqiSfB2pc8uefjEituxFPWZmVtniFl80CxhXYdpxpHOsViaNGDwuIi5oZz3lHF9k1hkeCTsYHF+UPAd8KiI2At4PnCRp1XbUaGZmzXN8ERARd0fEPfn+I8DjwCJn2TohwsysOxxfVEbSFsAywL3lrzkhwsysOxo55lQeX3QLC+OLJrMwvgjSh/rUPN8Oko4AXgesDtwOXFplHaXxRR/O93cGdpc01KxqxRedmRvmJRFR3pwaJmkN0oCJ/SPi1VaXY2Zmw+P4ooXLXRn4LXB0RExrdv5m+aCtmVl1ji9Ky1gGuBg4JyIubKA2MzPrIMcXJR8FtgMm5rpm5OHlZmbWA44vapHji8zMmuf4IjMz61uOL3J8kZlZ4Ti+iAWDKn4NLEk6zvWjiPhJO+sp5/gis87wSNjB4Pii5FFgm3wi75aky8O3pUYzM2ue44uAiPhXRAyda7Vste+L44vMzLrD8UWZpHXycPiHgeOHdjuWcnyRmVl3NNKcyuOLprIwvuh5FsYXzQD2B9bN8+0g6UZJs4AdgY1qrKM0vmhMvr8zaffaDGAKteOLPp1TKN4eEc828J4WEREPR8QmwFhgf0lvaGU5ZmY2fI4vKhMRj0i6ndR8L2p1OfX4oK2ZWXWOL0rLWHvo+Jek1YB3AXfVmsfMzDrH8UXJW4EbJd0G/An4Xh52bmZmPeD4ohZJegJ4sIVZRwJtPaerg/qpVuivevupVuivevupVuivettR67oRUXdEmZtTl0ma3kiuVBH0U63QX/X2U63QX/X2U63QX/V2s1bHFzm+yMyscBar5lQtvsjMzIrFqeTdd1qvC2hCP9UK/VVvP9UK/VVvP9UK/VVv12r1MSczMyscbzmZmVnhuDmZmVnhuDl1mKS9Jd0u6VVJVYdgSnq/pLskzZF0ZDdrLKtjdUlXSbonf12tynTfze/rTkk/HEryKGitoyX9Ptd6h6Qx3a208VrztCtL+l9JJ3ezxrIa6tYraZzSFQlulzRT0j5drrHm34ykZSVdkF+/sRc/95Ja6tV6aP7dnCnp6px80zONfh5J2ktS1Ppsa5WbU+fNBj5Min6qSNKSwI+BD5CCdD8u6W3dKW8RRwJXR8QGwNX58WtI2oYU8bQJsDGwObB9N4vM6taanQNMioi3AlsAj3epvlKN1gopDeVPXamqukbqfQ74VERsBLwfOEnSqt0orsG/mc8CT0XEWOBE4Phu1FauwVpvJWWRbkLK9Pxud6tcqNHPoxwGfghwYyfqcHPqsIi4szy8toItgDkRcV9E/Av4JbBH56uraA/g7Hz/bOBDFaas/aFTAAAEl0lEQVQJUn7iMqTrXy0NPNaV6l6rbq35j2qpiLgKICLmR8Rz3StxgUa+r0jaDHgD8Psu1VVN3Xoj4u6IuCfff4TU9Lt1LZlG/mZK38NFwHt6sYVPA7VGxB9Lfi+nAWt3ucZSjX4efYvURF/oRBFuTsWwFuk6UkPm5ud64Q0R8ShA/vr68gkiYirwR9IVhB8lXU/rzq5WmdStFdgQeFrSr5UuWjkp/2fYbXVrlbQE8H2g6nXHuqiR7+0CkrYg/bNybxdqg8b+ZhZMExEvA88AI7pSXZU6snp/358FruhoRbXVrTcHca8TEZd1qojF6iTcTpH0B+CNFV46KiJ+08giKjzXsTH+teptcP6xpLDcof/urpK0XURU3XXZquHWSvod3xbYFHgIuIAUVvyzdtRXqg21HgRcni8z077CqmhDvUPLWYOUvLJ/RLzajtoaWW2F58r/Zrr6d1VDw3VI+gTpenm92E2+oIwKzy2oN/8TdSLp76hj3JzaICJ2GuYi5gLrlDxeG1jkSrztUqteSY9JWiMiHs0fOpWOz+wJTIuI+XmeK4CtqHFcrYe1zgVujYj78jyX5Frb3pzaUOvWwLaSDgJWBJaRND8iOjJApg31Imll4LfA0RExrRN1VtHI38zQNHMlLQWsAvyd7mvo71vSTqR/DLaPiBfLX++ievWuRDrWPCX/E/VGYLKk3SNieruK8G69YrgJ2EDSmyQtA3wMmNyjWiaTrmhM/lppy+8hYHtJS0lamvRfXi926zVS603AapKGjoXsCNzRhdrK1a01IvaLiNERMYZ0/bRzOtWYGlC33vy7ejGpzgu7WBs09jdT+h72Aq6J3qQO1K017yb7KbB7RPRiwE6pmvVGxDMRMTIixuTf1WmkutvWmIZW5FsHb6StjLmkq/0+Rjo+A7AmaRfO0HS7AHeT9tkf1cN6R5BGZ92Tv66enx8PnJHvL0n6Q7qT9EF/QlFrzY/fS7qC8yzS9cSWKWqtJdNPBE4u+O/BJ4CXSBceHbqN62KNi/zNAMeSPighDdq5EJgD/AVYr4ffz3q1/iF/Pgx9Hyf3qtZG6i2bdgpppGFba3B8kZmZFY5365mZWeG4OZmZWeG4OZmZWeG4OZmZWeG4OZmZ2QKSzpT0uKTZDUy7naRbJL0saa+y116RNCPfmj41xs3JzMxK/TcpyLcRD5FOe/hFhdeej4hx+bZ7s0W4OZmZ2QKRYshek6QhaX1Jv5N0s6TrJL0lT/tARMwE2h5b5eZkZmb1nAb8e0RsRkovOaWBeZaTNF3SNEkVU/hrcbaemZlVJWlFYBvgwpJA4mUbmHV0RDwiaT3gGkmzIqLh1Ho3JzMzq2UJ4OmIGNfMTJGu8UVE3CdpCunKAA03J+/WMzOzqiLiH8D9kvYGUPKOWvNIWk3Ssvn+SNKVs5sKXHa2npmZLSDpfGACMJIURvsN4BrgVGAN0pWvfxkRx0ranJRMvxrpirh/i4iNJG1DCod+lbQRdFJENHWZGjcnMzMrHO/WMzOzwnFzMjOzwnFzMjOzwnFzMjOzwnFzMjOzwnFzMjOzwnFzMjOzwvk/aK90T9mskxwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1066310d0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot important coefficients\n",
    "coefs = pd.Series(lr.coef_, index = feat_names)\n",
    "print(\"Ridge picked \" + str(sum(coefs != 0)) + \" features and eliminated the other \" +  \\\n",
    "      str(sum(coefs == 0)) + \" features\")\n",
    "\n",
    "#正系数值最大的10个特征和负系数值最小（绝对值大）的10个特征\n",
    "imp_coefs = pd.concat([coefs.sort_values().head(10),\n",
    "                     coefs.sort_values().tail(10)])\n",
    "imp_coefs.plot(kind = \"barh\")\n",
    "plt.title(\"Coefficients in the OLS Model\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "season_1        2.951049e+12\n",
       "season_2        2.951049e+12\n",
       "season_3        2.951049e+12\n",
       "season_4        2.951049e+12\n",
       "mnth_1          4.095092e+14\n",
       "mnth_2          4.095092e+14\n",
       "mnth_3          4.095092e+14\n",
       "mnth_4          4.095092e+14\n",
       "mnth_5          4.095092e+14\n",
       "mnth_6          4.095092e+14\n",
       "mnth_7          4.095092e+14\n",
       "mnth_8          4.095092e+14\n",
       "mnth_9          4.095092e+14\n",
       "mnth_10         4.095092e+14\n",
       "mnth_11         4.095092e+14\n",
       "mnth_12         4.095092e+14\n",
       "weathersit_1   -9.926428e+14\n",
       "weathersit_2   -9.926428e+14\n",
       "weathersit_3   -9.926428e+14\n",
       "weekday_0      -4.050548e+14\n",
       "weekday_1      -2.864121e+14\n",
       "weekday_2      -2.864121e+14\n",
       "weekday_3      -2.864121e+14\n",
       "weekday_4      -2.864121e+14\n",
       "weekday_5      -2.864121e+14\n",
       "weekday_6      -4.050548e+14\n",
       "temp            2.816109e+03\n",
       "atemp           1.076680e+03\n",
       "hum            -1.885078e+03\n",
       "windspeed      -1.512276e+03\n",
       "holiday        -1.186426e+14\n",
       "workingday     -1.186426e+14\n",
       "yr              1.938977e+03\n",
       "dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coefs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "系数的值非常大。由于特征之间强相关，OLS模型的性能并不好"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "a70803d8-638c-bdfa-6897-aed8ce78f475",
    "_uuid": "2938aee78a9f61d6ca59345d6fa0793e2422f34c"
   },
   "source": [
    "**2* Linear Regression with Ridge regularization (L2 penalty)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('Best alpha :', 1.0)\n",
      "('cv of rmse :', array([  804.98946948,   801.06246693,   797.95313648,   822.15888399,\n",
      "        1078.04194503,  1680.07803178]))\n",
      "('RMSE on Training set :', 754.03666237615039)\n",
      "('RMSE on Test set :', 776.97536071338061)\n",
      "('r2_score on Training set :', 0.84293647640679858)\n",
      "('r2_score on Test set :', 0.85806008968776237)\n"
     ]
    }
   ],
   "source": [
    "#RidgeCV缺省的score是mean squared errors \n",
    "# 1. 设置超参数搜索范围，生成学习器实例\n",
    "# RidgeCV(alphas=(0.1, 1.0, 10.0), fit_intercept=True, normalize=False, scoring=None, cv=None, gcv_mode=None, store_cv_values=False)\n",
    "alphas = [0.01, 0.1, 1, 10, 100, 1000]\n",
    "ridge = RidgeCV(alphas = alphas,store_cv_values=True )\n",
    "\n",
    "# 2. 用训练数据度模型进行训练\n",
    "# RidgeCV采用的是广义交叉验证（Generalized Cross-Validation），留一交叉验证（N-折交叉验证）的一种有效实现方式\n",
    "ridge.fit(X_train, y_train)\n",
    "\n",
    "#通过交叉验证得到的最佳超参数alpha\n",
    "alpha = ridge.alpha_\n",
    "print(\"Best alpha :\", alpha)\n",
    "\n",
    "# 交叉验证估计的测试误差\n",
    "mse_cv = np.mean(ridge.cv_values_, axis = 0)\n",
    "rmse_cv = np.sqrt(mse_cv)\n",
    "print(\"cv of rmse :\",rmse_cv)\n",
    "\n",
    "#训练上测试，训练误差，实际任务中这一步不需要\n",
    "y_train_pred = ridge.predict(X_train)\n",
    "rmse_train = np.sqrt(mean_squared_error(y_train,y_train_pred))\n",
    "\n",
    "y_test_pred = ridge.predict(X_test)\n",
    "rmse_test = np.sqrt(mean_squared_error(y_test,y_test_pred))\n",
    "\n",
    "print(\"RMSE on Training set :\", rmse_train)\n",
    "print(\"RMSE on Test set :\" ,rmse_test)\n",
    "\n",
    "r2_score_train = r2_score(y_train,y_train_pred)\n",
    "r2_score_test = r2_score(y_test,y_test_pred)\n",
    "print(\"r2_score on Training set :\" ,r2_score_train)\n",
    "print(\"r2_score on Test set :\" ,r2_score_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "_cell_guid": "1fb3f0d6-a070-9ce5-7cc3-01a7e26faa4f",
    "_uuid": "659084128a0ef75a5936912a5f64134774b29664",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ridge picked 33 features and eliminated the other 0 features\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAacAAAEICAYAAAD7pTujAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmcXFWZ//HPV2QPGiER2SObGdZA\nwpIBNGAExQVQGLYRgg6OC4O+/LEJDGYGURQ3FBEBkS2DCgKCICGCISGEkASSdCI7CUMkA4RNVIhI\nnt8f57R9U1R1V3V3bd3f9+vVr9y699x7n6pU+uSc+9znKiIwMzNrJW9pdgBmZmal3DmZmVnLcedk\nZmYtx52TmZm1HHdOZmbWctw5mZlZy3HnZG1N0nskPSDpFUknSlpb0s2SXpZ0raSjJd1exXFOl3Rp\nI2LuJobNJf1Z0mr9dLyJkq7uj2NVca5uP2dJUyX9WyNi6Q+1xCspJG1d75gGG3dO1hCSjpI0J//y\nXSbpt5L27odDnwJMjYj1IuIHwKHAhsAGEXFYREyKiP17OkhEfD0i+vzLU9KI/MvqrbXuGxH/GxFD\nIuKNXpx3nKSlte5Xw/Evl/S3/Pf3gqQpkkZ2bq/2c65DXBPz531iyfov5fUTGx2T9Q93TlZ3kr4M\nfB/4Oqnj2By4EDioHw6/BbCo5PUjEfH3fji2repbETEE2AT4I/DTJsfT6RHg2JJ1x+T11qbcOVld\nSXo78N/AFyLi+oj4S0S8HhE3R8TJuc2akr4v6en8831JaxaO8RFJ8yS9JOkeSTvl9XcC+wIX5P/R\nXwOcBRyeX39a0gRJdxeOtX3+X/8Lkp6RdHpev8oUmKQ987lekjRf0rjCtqmSzpY0I08n3i5pWN48\nLf/5Uo5hrKStJd2VpxqXS/pFhc9qlVFXD+cp7rcu8Ftg43zOP0vaOG9eQ9KVef9FksYU9ttY0q8k\nPSdpcenoo5KIeBX4JTCqcKzSz/kDkh7K7/kCQIVtq0n6Tv4sFks6oeR9v13ST/MI+4+SvtbDVOds\nYB1J2+f9twfWzuuLn9Pxkh7Lf/c3FT6jbuPN2z8l6UFJL0qaLGmLaj4r6z13TlZvY4G1gBu6aXMG\nsCfpl93OwO7AmQCSdgUuA/4d2AD4CXCTpDUjYj9gOnBCng47kjQ6+0V+vcr/7CWtB/wOuA3YGNga\nuKM0GEmbALcAXwPWB04CfiVpeKHZUcBxwDuBNXIbgPfmP4fmGGYCZwO3A+8ANgV+2M1nUarSef4h\nIv4CfAh4Op9zSEQ8nTd/DPg5MBS4Cbggv8e3ADcD80kjofcDX5J0QE8B5c7wSOCxCtuHAb8i/R0O\nAx4H9io0OT7HOwrYFTi45BBXAH8n/f3sAuwP9DTlehVptARpFHVlSUz7Ad8A/gXYCHiS9Ln0GK+k\ng4HTgY8Dw0nfuWt6iMf6yJ2T1dsGwPIeptmOBv47Ip6NiOeA/wI+mbcdD/wkImZFxBsRcQWwgtSZ\n1eojwP9FxHci4rWIeCUiZpVp96/ArRFxa0SsjIgpwBzgwEKbn0XEI+VGEWW8Tppu3Dif9+5u2paq\n5Tzl3J3fxxukX+A75/W7AcMj4r8j4m8R8QRwCXBEN8c6SdJLwCvA3nT9HZU6EPhDRFwXEa+TpnT/\nr7D9X4DzI2JpRLwInNu5QdKGpI7rS3mU/SzwvR7iArgaOFLS6rltaSLI0cBlEXF/RKwAvgKMlTSi\ninj/HfhGRDyYv8dfB0Z59FRf7pys3p4Hhqn7BIGNSf+T7fRkXgfpl/r/y9NrL+VfjpsVttdiM9L/\ninuyBXBYyTn3Jv2Pu1Pxl9dfgSHdHO8U0jTRfXlq7VM1xFzLearZf638d7EFaRqw+B5PJ10TrOTb\nETEUGAG8CrynQruNgac6X0SqLv1Upe0ly1sAqwPLCnH9hDRyrCgi/pc0kvs68GhEPFXSZJXvWET8\nmfTd3KSKeLcAzi/E8wLp73OT7mKyvqk5o8isRjOB10hTN9dVaPM0qyY2bJ7XQfolcU5EnNMPsTxF\nmo6qpt1VEXF8L87xpjL/EfF/pBEgShmKv5M0LSLKTov1Uq2PF3gKWBwR29R8ooj/lfRF4ApJv8mj\nuqJlpP8IACBJxdd5+6aF18VtT5FGxsN6kdRyJWkK+Lgy2zq/Y50xrUsa1f+xing7v4OTaozH+sAj\nJ6uriHiZlKTwI0kHS1pH0uqSPiTpW7nZNcCZkobn+f+z6JqWuQT4rKQ9lKwr6cP5+lGtfgO8SynN\neE1J60nao0y7q4GPSjogX7xfSylVe9MybUs9B6wEtuxcIemwwr4vkjqSmtPFe/AMsIFSAko17gP+\nJOlUpXvDVpO0g6Tdqtk5T3U+DXymzOZbgO0lfTyP0k4E3lXY/kvgi5I2kTQUOLVw3GWk63PfkfQ2\nSW+RtJWk91UR1i9I16d+WWbb/wDHSRqllGzzdWBWRCypIt6LgK8UEi7eLumwKuKxPnDnZHUXEd8F\nvky64Pwc6X+iJwA35iZfI13TWQB0APfndUTEHNKo4wLSL/bHgAm9jOMV4APAR0nTXY+Ssv1K2z1F\nSnM/vRDvyVTx7yUi/gqcA8zI00B7kq7vzJL0Z1JSwhcjYnFv3kM3532I1Mk/kc/b7bRnvgb1UdI1\nrMXAcuBSoNrODeA84BQVMivzsZcDh5GuJT0PbAPMKDS5hNQBLQAeAG4lJUB0dtjHkJI//kD6O7+O\nVadUK72nVyPid2VGckTEHcB/khIflgFbka9j9RRvRNwAfBP4uaQ/AQtJ18WsjuSHDZpZM0n6EHBR\nRDjBwP7BIycza6g8jXigpLfmtP2v0v2tBjYIeeRkZg0laR3gLmAkKevvFtJU55+aGpi1FHdOZmbW\ncjytZ2ZmLcf3OfXSsGHDYsSIEc0Ow8ysrcydO3d5RAzvqZ07p14aMWIEc+bMaXYYZmZtRdKTPbfy\ntJ6ZmbUgj5zMWsSI025pdghmVVly7ofrfg6PnCro4fkxZmZWR4N25CTpbNKjHM7Pr88h1Sc7hFTe\nZBSwXfMiNDMbvAbzyOmn5Ec75wevHUGqULw7cEZEvKljkvQZSXMkzXnuuecaGqyZ2WAyaDunXI34\neUmdT9p8gFT08b5KRTkj4uKIGBMRY4YP7zET0szMemnQTutll5IqXL+L9BwYgL80LRozMwPcOd0A\n/DfpyZtHAfs0NxwbzBqRAWXWLgZ15xQRf5P0e+CliHgjPQDTzMyabVB3TjkRYk/Sg8aIiKnA1CaG\nZGZmDOKECEnbkZ6qekdEPNrseMzMrMugHTlFxB+ALZsdh5mZvVnbdk6ShgJHRcSFzY7FrD+4fJG1\nomYl6rTztN5Q4PPNDsLMzPpfO3dO5wJbSZon6TxJJ0uaLWmBpP8CkDRC0kOSLpW0UNIkSeMlzZD0\nqKTdc7uJkq6SdGdef3xT35mZ2SDXzp3TacDjETEKmAJsQyo9NAoYLem9ud3WwPnATsBI0v1MewMn\nAacXjrcT8GFgLHCWpI1LT+jyRWZmjdHOnVPR/nSVILqf1Altk7ctjoiOiFgJLCJl5wXQAYwoHOPX\nEfFqRCwHfk/q6Fbh8kVmZo3RtgkRJQR8IyJ+sspKaQSworBqZeH1SlZ9/1FyzNLXZnXlChFmXdp5\n5PQKsF5engx8StIQAEmbSHpnjcc7SNJakjYAxgGz+y1SMzOrSduOnCLi+ZzYsBD4LfA/wMxcgujP\nwL8Cb9RwyPuAW4DNgbMj4ul+DtnMzKrUtp0TQEQcVbLq/DLNdii0n1BYXlLcBjwSEZ/pz/jMzKx3\n2nlaz8zMBqi27Zwknd5zq+pExMSI+HZ/Hc/MzPqmnaf1Tge+3uwgzPqLyxdZrQZyhmdbjJwk3Shp\nrqRF+UbYc4G1c3WISbnNv0q6L6/7iaTV8vo/S/pm3v93knaXNFXSE5I+lttMkPRrSbdJeljSV5v4\nds3MBr226JyAT0XEaGAMcCJwHvBqRIyKiKMl/RNwOLBXrhjxBnB03nddYGre/xXga8AHgENIT8Ht\ntHveZxRwmKQxDXhfZmZWRrtM650o6ZC8vBld1R86vR8YDczOqeRrA8/mbX8DbsvLHcCKiHhdUmmF\niCkR8TyApOtJJY7mFE8i6TPAZwA233zzvr8rMzMrq+U7J0njgPHA2Ij4q6SpwFqlzYArIuIrZQ7x\nei5XBIUKERGxUlJNFSIi4mLgYoAxY8a4goSZWZ20fOcEvB14MXdMI0mPVQd4XdLqEfE6cAfwa0nf\ni4hnJa0PrBcRT9Zwng/k/V4FDgY+1Z9vwqwnA/nitlmt2uGa023AWyUtAM4G7s3rLwYWSJqUn2p7\nJnB7bjcF2KjG89wNXAXMA34VEXN6aG9mZnWirhmvwUvSBGBMRJxQ7T5jxoyJOXPcf5mZ1ULS3Ijo\nMeGsHUZOZmY2yLTDNae6i4jLgcubHIaZmWUeOZmZWctp+MgpX9+5vfORFJKWkK73LO/n89xKeiQ7\nwFERcWEP7W8jZQLeHREf6c9YzKrh8kWJsxYNmjNymgBs3B8HKrlPaRURcWBEvAQMBT5fxeHOAz7Z\nH3GZmVnf9Ng5STpF0ol5+XuS7szL75d0taT9Jc2UdL+kawtPoz1L0mxJCyVdrORQUgmiSbkG3tr5\nNP+R9+/I9zIhaV1Jl+VjPCDpoLx+Qj7PzaTU8Y0kTcvHWyhpn9xuiaRhwLnAVnn7eZXeZ0TcQSpv\n1N1n8RlJcyTNee6553r66MzMrJeqGTlNA/bJy2OAIZJWJ5X36SDdXzQ+InYllfv5cm57QUTsFhE7\nkMoJfSQirsttjs518V7NbZfn/X8MnJTXnQHcGRG7AfsC50laN28bCxwbEfuRpu4m55p6O5PuUyo6\nDXg8n+/kaj6USiLi4ogYExFjhg8f3pdDmZlZN6q55jQXGC1pPVLpn/tJndQ+wE3AdsCMXNNuDWBm\n3m9fSacA6wDrA4uAmyuc4/rCuT6el/cHPiaps7Nai/QIdUh18F7Iy7OBy3KHeWNElHZOZmbWZnrs\nnHKR1CXAccA9wALSSGYrYDGpoziyuI+ktYALSYkOT0mayJvr4RWtyH++UYhJwCci4uGSY+8B/KUQ\n3zRJ7wU+DFwl6byIuLKn92XWapwIYNal2oSIaaTptmnAdOCzpOmze4G9JG0NIGkdSdvS1REtz9eg\nDi0c6xVgvSrOOZl0LUr52LuUayRpC+DZiLgE+Cmwa0mTas9nZmYtotrOaTqpVt3MiHgGeA2YHhHP\nkbLvrsk17e4FRuYsuUtI16RuJE29dbocuKgkIaKcs4HVSfXzFubX5YwD5kl6APgEcH5xY34Mxoyc\nLFExIULSdOBa4P2Slko6oJvYzMysjlxbr5dcW8/MrHaurWdmZm1rUNXWk7Qj6bEYRSsiYo9mxGNm\nZuUNmM5J0lAKZYryE3RPKpYiiogOYFSF/fcDvk1Kh58LfDoi/l7vuM06DabyRc5MtJ4MpGm9assU\nvYmktwBXAEfkm4afBI7tx9jMzKwGLdU5SRoh6SFJl+bsukmSxkuaIelRSbtLmpjLGk2V9ERnaSXK\nlykaIum6fMxJnWnpZWxAmt57JL+eQsr8K43P5YvMzBqgpTqnbGtSOvhOwEhSeaK9SfdZnZ7bjAQO\nAHYHvpqrQ5QrU7QL8CVSFYstgb0qnHM5sLqkzgySQ4HNShu5fJGZWWO0Yue0OCI6ImIlqeTRHZHy\n3TuAEbnNLRGxIj9m41lgwwrHui8iluZjzSvsv4p8/COA70m6j3Tjrq83mZk1SSsmRKwoLK8svF5J\nV7zFNsWSR90dq7t2RMRMcoFbSfsD21YfslnfOUnArEsrjpx6q09liiS9M/+5JnAqcFE/xWVmZjUa\nMJ1TtWWKunGypAdJhW1vjog7+zdCMzOrlssX9ZLLF5mZ1c7li8zMrG21YkJEXUm6AXh3yepTI2Jy\nM+Ix6zRYKkQ48cOqMeg6p4g4pLvtkm4CtsyVIszMrAk8rVcg6ePAn5sdh5nZYNewzknSupJukTQ/\nZ9QdLmm0pLskzZU0WdJGue3xkmbntr+StE5ef1jed76kaXndWpJ+JqlD0gOS9s3rJ0i6XtJtufTR\nt3qIbwjwZeBr3bRx+SIzswZo5Mjpg8DTEbFznjK7DfghcGhEjAYuA87Jba+PiN0iYmfgQeDTef1Z\nwAF5/cfyui8ARMSOwJHAFZI6HxM/Cjgc2BE4XNKbShIVnA18B/hrpQYuX2Rm1hiN7Jw6gPGSvilp\nH1Ltuh2AKZLmAWcCm+a2O0iaLqkDOBrYPq+fAVwu6Xhgtbxub/IzmiLiIVJF8c7qDndExMsR8Rrw\nB2CLcoFJGgVsHRE39N/bNTOz3mpYQkREPCJpNHAg8A1S5e9FETG2TPPLgYMjYr6kCcC4fIzPStoD\n+DAwL3cqlSqNQ/Xli8YCoyUtyW3eKWlqRIyr7t2Z9Z2z2My6NPKa08bAXyPiatJD/fYAhksam7ev\nLqlzhLQesCxXGz+6cIytImJWRJxFqiS+GTCts42kbYHNgYdriS0ifhwRG0fECNJI7BF3TGZmzdPI\nVPIdgfMkrQReBz5Hqvz9A0lvz7F8n1SJ/D+BWaQpug66auadJ2kb0mjpDmA+8BBwUZ4C/DswISJW\nVH50k5mZtTqXL+olly8yM6udyxeZmVnbGnQVIiTNAtYsWf3JiOhoRjxmnVq1fJETNawZBkznJGko\ncFREXJhfjwNOioiPFNtFxB4V9r8ceB/wcl41ISLm1S1gMzOraCBN6w0FPt/HY5wcEaPyjzsmM7Mm\naanOSdIISQ9JujSXKZokabykGbkE0e6SJkq6TNJUSU9IOjHvfi6wlaR5hYcNDpF0XT7mJPUxhc/l\ni8zMGqOlOqdsa+B8YCdgJHAU6d6jk4DTc5uRwAHA7sBX8/1QpwGP51HPybndLsCXgO2ALYG9ejj3\nOZIWSPpeflz7Kly+yMysMVqxc1ocER0RsZJ0z9MdkfLdO4ARuc0tEbEiIpYDzwIbVjjWfRGxNB9r\nXmH/cr5C6vR2A9YHTu3zOzEzs15pxYSIYsmhlYXXK+mKt9qyRNW2IyKWde4j6WekkZpZwzgrzqxL\nK46ceusVuipJ1KzwuA4BBwML+ykuMzOrUSuOnHolIp7PiRMLgd8Ctd40MknScFJppHnAZ/s7RjMz\nq47LF/WSyxeZmdXO5YvMzKxtDZhpvWpJugF4d8nqUyNicjPiMevUrPJFTsSwVtTwkZOkCfnZTp2v\nl0gaVofz3CppaP75R+WIiDikUAViVESMAp6RNFPSonyf0+H9HY+ZmVWvGdN6E4CNe2pUDUndpYYf\nGBEvUV1Zo78Cx0TE9sAHge/nWn1mZtYEPXZOkk7pLBGUKyfcmZffL+lqSfvnUcf9kq6VNCRvP0vS\n7FyG6GIlhwJjSJlx8yStnU/zH3n/Dkkj8/7r5jJFsyU9IOmgvH5CPs/NwO2SNpI0LR9voaR9crvO\nEVm5skariIhHIuLRvPw06cbeN5WAcPkiM7PGqGbkNA3YJy+PIdWrW51UUqgDOBMYHxG7AnOAL+e2\nF0TEbhGxA7A28JGIuC63OTpPqb2a2y7P+/+YrptfzwDujIjdgH1JT8FdN28bCxwbEfuRyhtNztNz\nO5PSwIvKlTWqSNLuwBrA46XbXL7IzKwxqkmImAuMlrQeqeLC/aROah/gJlLduhm5puoawMy8376S\nTgHWIZUDWgTcXOEc1xfO9fG8vD/wMUmdndVawOZ5eUpEvJCXZwOX5Q7zxr5UE8834l5F6vhW9vY4\nZmbWNz12ThHxuqQlwHHAPcAC0khmK2AxqaM4sriPpLWAC4ExEfGUpImkzqWSzjJDxRJDAj4REQ+X\nHHsP4C+F+KZJei/wYeAqSedFxJU9va9Skt5GunH3zIi4t9b9zfrKWXNmXapNiJhGmm6bBkwnVU+Y\nB9wL7CVpawBJ60jalq6OaHm+BnVo4VjVlhmaTLoWpXzsXco1krQF8GxEXAL8FNi1pEmP55O0BnAD\ncGVEXFtFbGZmVkfVdk7TgY2AmRHxDPAaMD0iniNl310jaQGpsxqZs+QuIV2TupE09dbpcuCikoSI\ncs4GVgcW5JJEZ1doNw6YJ+kB4BOkx238Q0Q8T5p2XFgpIQL4F+C9wIQc1zxJo7qJzczM6sjli3rJ\n5YvMzGrn8kVmZta2BlX5Ikk7krLxilZExB7NiMesqB7li5xkYe2qrUZOpaWIJI2T9JsaDvE+YAjp\nfqjx+d6nPfINwj+Q9FguX1SaVGFmZg3UVp0T1ZUi6s4MYDzwZMn6DwHb5J/PkG4GNjOzJmlG4dcR\nkh6SdGnOoJskaXx+UOCjknaXNDGXLpoq6YnO8kmUL0U0RNJ1+ZiTOlPPy4mIByJiSZlNB5HSyCPf\n4zQ035BbGrvLF5mZNUCzRk5bk1K+dwJGkkoQ7U26l+r03GYkcACwO/DVXAGiXCmiXYAvkSpVbAns\n1Yt4NgGeKrxemtetwuWLzMwao1kJEYsjogNA0iLgjogISR3ACNINvrdExApghaRngQ0rHOu+iFia\njzUv7393jfGUG205x94ayskLZl2aNXJaUVheWXi9kq4Os9imWNaou2N11647S4HNCq83BZ7uxXHM\nzKwftFtCRLWlj2p1E3BMztrbE3g5IpbV4TxmZlaFtuqcqixFVJGkEyUtJY2MFki6NG+6FXgCeIxU\ndqkvGYFmZtZHLl/USy5fZGZWO5cvMjOzttVy5YskTQVOioiKwxJJE0jPijqhwvYbgHeXrD41IiZ3\nc8x/ASaSsvTmR8RRtUVu1jf9Xb7I2X/Wzlquc+oPEXFILe0lbQN8BdgrIl6U9M76RGZmZtXo87Se\npFM6KzhI+p6kO/Py+yVdLWl/STMl3S/p2vzwQSSNlnSXpLmSJpdWZJD0FklXSPpafn2cpEck3UXh\nRltJH5U0S9IDkn4nacO876OShheO9ZikYRXexvHAjyLiRYCIeLavn4uZmfVef1xzmgbsk5fHkMoJ\nrU6q+NABnEkqsrorMAf4ct7+Q+DQiBgNXAacUzjmW4FJwCMRcWbuuP6L1Cl9gFQNotPdwJ4RsQvw\nc+CUiFgJXA0cnduMJ03VLa/wHrYFts0llO6V9MFyjVy+yMysMfpjWm8uMFrSeqQbYu8ndVL7kO4f\n2o6U/g2wBjATeA+wAzAlr18NKN5X9BPglxHR2WHtAUzNT95F0i9IHQqktPBf5A5sDWBxXn8Z8Gvg\n+8CngJ918x7eSir6Oi4fb7qkHfITff8hIi4GLoaUrdfzR2NmZr3R584pIl6XtAQ4DrgHWADsC2xF\n6iimRMSRxX3yc5UWRcTYCoe9B9hX0nci4rXOU1Vo+0PguxFxk6RxpKQGIuIpSc9I2o/UuR1dYX9I\nFSLujYjXgcWSHiZ1VrO72cesXzmBwaxLf6WSTyMVbZ0GTAc+S6qPdy+wl6StASStI2lb4GFguKSx\nef3qkrYvHO+npBtjr5X0VmAWME7SBnlK8LBC27cDf8zLx5bEdSlpeu+XEfFGN/HfSOpQydeltiXd\nlGtmZk3QX53TdGAjYGZEPAO8BkzP03ATgGskLSB1ViMj4m/AocA3Jc0ndWT/XDxgRHyXNEV4FfAM\naUQ0E/hdXt9pIqkTmw6UXlO6ifRwwe6m9AAmA89L+gPwe+DkXI3CzMyaYEBXiJA0BvheROzTY+Ma\nuUKEmVntqq0QMSDvcwKQdBrwObq/1mRmZi2o5coX5SflLqyh/eWSDs3Ll0raDiAizo2ILSLibkkT\nJF0g6Yz8FN3izxn1ei9mZtY7A2rkFBH/1sP2c1j1fiqzPuuvskPO1jPr0nIjp2w1SZdIWiTpdklr\nSxqVb5BdIOkGSe8o3UnS1Hydqd4VJczMrI5atXPahlROaHvgJeATwJWk4q07kSpPfLXSzg2oKGFm\nZnXUqp3T4oiYl5fnkm7oHRoRd+V1VwDv7Wb/f1SUyGnrvyhs2xSYLKkDOBnovL/qMuCYvFy2ooTL\nF5mZNUardk4rCstvAEN7cYzuKkpcEBE7Av8OrAWpogRQrCjx2zcdMOLiiBgTEWOGDx/ei5DMzKwa\n7ZIQ8TLwoqR9ImI68Engrm7azwLOl7QB8CdSRYn5eVs1FSWu6qGihNk/OJHBrP+1S+cEqSO5SNI6\npNJCx1VqGBHLJE0kVZRYRqoosVrePJFUUeKPpIoVxYcS3kSazuupooSZmdXRgK4QUataKkq4QoSZ\nWe0GfYWIWrmihJlZ62jVhIiGK1aUaHYsZmaDnTsnMzNrOQNmWk/SUOCoiLgwvx4HnBQRH6ly/0mk\nJ/i+DtwH/Ht++KDZKvqrXFEpZ/2ZdRlII6ehwOf7sP8kYCSwI7A20G2dPjMzq5+W6pxyRfKHcnXx\nhZImSRovaUaufbe7pImSLst19J6QdGLe/Vxgq1xp/Ly8boik6/IxJ0lSpXNHxK2RkUZOm5aJzxUi\nzMwaoKU6p2xr4HxgJ9JI5ihgb9Jj4E/PbUYCBwC7A1/Nj24/DXg8IkZFxMm53S7Al0i19bakUAC2\nknysTwK3lW5zhQgzs8Zoxc5pcUR05GKsi4A78mimAxiR29wSEStyYdZngQ0rHOu+iFiajzWvsH93\nLgSm5UoUZmbWBK2YEFGsq7ey8HolXfGW1t6r9D6qbQeApK8Cw0k198zKcuKCWf21YufUW68A6/V2\nZ0n/RpoqfH8eaZmZWZO04rRer0TE88CMnEhxXo87vNlFpOnBmTmp4qz+jdDMzKrl2nq95Np6Zma1\nq7a23oAZOZmZ2cAxkK45VUXSDaz6mAxIj3+f3Ix4zMzszQZM51Rt+aKIOKTC/tPpSqh4JykN/eD6\nRWztoF6lispxFqBZl4E0rden8kURsU++gXcU6SGF1/dbZGZmVpOW6pyaWb6oEMN6wH7AjWW2uXyR\nmVkDtFTnlDW1fBFwCKkqxZ9KN7h8kZlZY7Ri59Ts8kVHAtf05Q2YmVnftGJCRDPLF21AGo2VTZqw\nwcdJCmbN0Yojp97qU/mi7DDgNxHxWj/EY2ZmvTRgOqd+KF8EcASe0jMzazqXL+olly8yM6udyxeZ\nmVnbasWEiLpy+SIrp5GVICpx8oVZl7YaOUkaKunzhdfjJP2mhv1PAHYEdgbGFypCDJO0IP/cI2nn\n/o/ezMyq1VadE30sUQTMAMb8ngZ+AAAPW0lEQVQDT5asXwy8LyJ2As4GLu7DOczMrI8a3jk1s0RR\nRDwQEUvKrL8nIl7ML+8FNq0Qu8sXmZk1QLNGTs0uUdSdTwO/LbfB5YvMzBqjWZ1Ts0sUlSVpX1Ln\ndGpvj2FmZn3XrGy9ppUoqkTSTsClwIfyDb02iDhTzqy1tFtCRH+UKHoTSZuTnt/0yYh4pL+Pb2Zm\ntWmrzqmvJYoknShpKSnhYYGkS/Oms4ANgAtzsoVLP5iZNZHLF/WSyxeZmdXO5YvMzKxtDcjyRS5R\nZLVohdJF4KQMs6IBM3IqljaKiENI9z4t7SxR1FPHJOkESY9JCknDGhGzmZmVN2A6J+pX2sjMzBqs\npTqnVixtVBKfyxeZmTVAS3VOWcuWNnL5IjOzxmjFzqklSxuZmVnjtGK2XsuVNrKBzVlyZq2nFUdO\nvVWX0kZmZtZ4A6ZzqmNpIzMzazCXL+olly8yM6udyxeZmVnbGnQJAi5tZK1SrqiUEzPMurTVyKlY\noii/HifpNzXsfwKwI7AzML6ztBHwpKSZklZIOqn/Izczs1q0VedE/UoUvQCcCHy7D8c2M7N+0vDO\nqRVLFEXEsxExG3i9h9hdvsjMrAGaNXJq2RJF3XH5IjOzxmhW5+QSRWZmVlGzsvVcosiaxllxZq2v\n3RIiXKLIzGwQaKvOqV4liiS9K6//MnCmpKWS3tavwZuZWdVcvqiXXL7IzKx2Ll9kZmZta0AmD/Sm\nRJGkc4BjgHdExJB6xjdYtWrZoFbhRA2zLgOyc4qIQ3qx283ABcCj/RyOmZnVqGHTepLWlXSLpPk5\noeFwSaMl3SVprqTJkjbKbY+XNDu3/ZWkdfL6w/K+8yVNy+vWkvQzSR2SHpC0b14/QdL1km7LlSe+\n1V18EXFvRCyr9+dgZmY9a+Q1pw8CT0fEzhGxA3Ab8EPg0IgYDVwGnJPbXh8Ru0XEzsCDwKfz+rOA\nA/L6j+V1XwCIiB2BI4ErJK2Vt40CDicVez1c0mZ9eQMuX2Rm1hiN7Jw6gPGSvilpH2AzYAdgiqR5\nwJmkFG+AHSRNl9QBHA1sn9fPAC6XdDywWl63N3AVQEQ8RCrqum3edkdEvBwRrwF/ALboyxtw+SIz\ns8Zo2DWniHhE0mjgQOAbwBRgUUSMLdP8cuDgiJgvaQIwLh/js5L2AD4MzJM0CqhY6BVXj2gpvuBv\nZtVq5DWnjYG/RsTVpEdT7AEMlzQ2b19dUucIaT1gWS72enThGFtFxKyIOAtYThp9TetsI2lbYHPg\n4Qa9LTMzq4NGjiR2BM6TtJL0aIrPAX8HfiDp7TmW75MKwf4nMIs0RddBV8mi8yRtQxot3QHMBx4C\nLspTgH8HJkTEim6enFFWTpg4ClgnV4u4NCIm9v7tmplZb7lCRC+5QoSZWe1cIcLMzNrWoEsQkDQL\nWLNk9ScjoqMZ8ZiZ2Zs1vHPK2Xe3R8TT+fUSYEx+qGB/nudW0jUkgKMi4kKAiNijTNstJM0lpaev\nDvwwIi7qz3jagcsLNZezGc26NGNabwKwcX8cSFLFzjUiDoyIl4ChwOd7ONQy4J8jYhQpi/C0nF1o\nZmZN0GPnJOkUSSfm5e9JujMvv1/S1ZL2lzRT0v2SrpU0JG8/K5cgWijpYiWHAmOASZLmSVo7n+Y/\n8v4dkkbm/deVdFk+xgOSDsrrJ+Tz3AzcLmkjSdPy8RbmG3yRtETSMOBcYKu8vewzoCLibxHReU/U\nmtV8LmZmVj/V/BKeBuyTl8cAQ/L9R3uT0rzPBMZHxK7AHNID+wAuyCWIdgDWBj4SEdflNkdHxKiI\neDW3XZ73/zFwUl53BnBnROwG7EtKI183bxsLHBsR+5Gm7ibnUc/OwLyS+E8DHs/nO7nSm5S0maQF\nwFPANzunHUvauHyRmVkDVNM5zQVGS1qPVHFhJqmT2gd4FdiO9HTaecCxdJUI2lfSrHz/0X50lSAq\n5/rCuUbk5f1J02vzgKnAWqQbbAGmRMQLeXk2cJykicCOEfFKFe/pTSLiqYjYCdgaOFbShmXauHyR\nmVkD9JgQERGv56SF44B7gAWkkcxWwGJSR3FkcZ9cePVCUqLDU7njWIvKOqfUiiWGBHwiIlap9pDL\nF/2lEN80Se8llTS6StJ5EXFlT++rkoh4WtIiUud7XW+P0458Qd7MWkW111amkabbpgHTgc+Sps/u\nBfaStDWApHVyCaHOjmh5vgZ1aOFYr9BV8aE7k0nXopSPvUu5RpK2AJ6NiEuAnwK7ljTp8XySNu28\n/iXpHcBeuASSmVnTVNs5TQc2AmZGxDPAa8D0iHiOlH13Tb5ecy8wMmfJXUK6JnUjaeqt0+WkckPF\nhIhyzialdS+QtDC/LmccqQjsA8AngPOLGyPiedK048JKCRHAPwGzJM0H7gK+7fuezMyax+WLesnl\ni8zMaufyRWZm1rYa+ciMWyUNraH9iDyd158x7JinE4s/s8q0+3N/ntfMzGrTyIcNHtioc3UTQwfp\n0e1tz6WGBh5nS5p16beRUxWVJJZIGpZHRA9KukTSIkm3FzLlRkuaL2km8IXCsbeXdF8e6SyQtE0+\nzkOSrsjrrpO0TuE4d0maK2mypI3y+q0k3ZbXTy9Uo3i3UpWL2ZIqJV6YmVmD9Oe0XneVJKaXtN0G\n+FFEbA+8RMqyA/gZcGKZR7d/Fjg/V4EYAyzN698DXJxvnv0T8Pl8zh8Ch0bEaOAy4Jzc/mLgP/L6\nk0j3YkHK8Ptxrkbxf5XeoCtEmJk1Rn92Tt1VkijtnBZHxLzCfiOUnoY7NCLuyuuvKrSfCZwu6VRg\ni0LZo6ciYkZevprUEb4H2AGYkqtLnAlsmu+3+mfg2rz+J6T0eEj3NV1T5ryrcIUIM7PG6LdrTj1U\nkniwpPmKwvIbpNp7AsrmtUfE/+TEhQ8DkyX9G/BEmfaRj7OodPQl6W3AS3n0VfY03b5BMzNrmP5O\niOisJPEp0g243wXmRkTkQg8VRcRLkl6WtHdE3A0c3blN0pbAExHxg7y8E6lz2lzS2IiYCRwJ3E2q\n7DC8c32e5ts2IhZJWizpsIi4Nlee2Cki5gMzgCNIo6+jaQO+eG5mA1l/p5KXrSRRw/7HAT/KCRGv\nFtYfDizM03Ejgc7aeQ+SirQuANYnXTf6G6lc0jdzxYd5pOk8SB3Pp/P6RcBBef0XgS9Img28vZY3\nbGZm/a9tK0RIGgH8Jj+So+FcIcLMrHauEGFmZm2rYTfh9reIWELKyjMzswFmwIyc6lHuyMzMmqNt\nR07tzuWHrJQzMM26DJiRU7ZaaVkkSVMljQHI5ZOW5OUJkm6UdHNOMT9B0pclPSDpXknrN/WdmJkN\nYgOtc6pUFqmSHYCjgN1JJY7+GhG7kCpSHFPa2OWLzMwaY6B1Tm8qi9RD+99HxCv5ib4vAzfn9R3l\n9nX5IjOzxhhonVNpWaS3An+n632u1U37lYXXK/H1ODOzphkMv4CXAKOB+0iVI1qCL36bmVU20EZO\n5Xwb+Jyke4BhzQ7GzMx61rbli5rN5YvMzGpXbfkid069JOk54Mk+HmYYsLwfwmkUx1tfjre+2ine\ndooVaot3i4joMaPMnVMTSZpTzf8gWoXjrS/HW1/tFG87xQr1iXcwXHMyM7M2487JzMxajjun5rq4\n2QHUyPHWl+Otr3aKt51ihTrE62tOZmbWcjxyMjOzluPOyczMWo47pzqSdFh+fMfKzsd25PUjJL0q\naV7+uaiwbbSkDkmPSfqBJOX160uaIunR/Oc7GhVv3vaVHNPDkg4orP9gXveYpNMK698taVaO9xeS\n1ujveEvimyjpj4XP9MDext5orRJHKUlL8ndxnqQ5eV3Z76GSH+T3sEDSrg2I7zJJzxYfMtqb+CQd\nm9s/KunYBsfbkt9bSZtJ+r2kB/PvhC/m9Y37fCPCP3X6Af4JeA8wFRhTWD8CWFhhn/uAsYCA3wIf\nyuu/BZyWl08DvtnAeLcD5gNrAu8GHgdWyz+PA1sCa+Q22+V9fgkckZcvAj5X5896InBSmfU1x97g\n70hLxFEhtiXAsJJ1Zb+HwIH5+ypgT2BWA+J7L7Br8d9SrfEB6wNP5D/fkZff0cB4W/J7C2wE7JqX\n1wMeyTE17PP1yKmOIuLBiHi42vaSNgLeFhEzI/3NXgkcnDcfBFyRl68orO833cR7EPDziFgREYuB\nx0jPwNodeCwinoiIvwE/Bw7Ko739gOvqGW+Vaoq9CfG1ShzVqvQ9PAi4MpJ7gaH5+1w3ETENeKGP\n8R0ATImIFyLiRWAK8MEGxltJU7+3EbEsIu7Py68ADwKb0MDP151T87xb6am7d0naJ6/bBFhaaLM0\nrwPYMCKWQfriAO9sXKhsAjxVJq5K6zcAXoqIv5esr7cT8pTCZYVpz1pjb7RWiaOcAG6XNFfSZ/K6\nSt/DVnkftcbXCnG39PdW0ghgF2AWDfx8B8MjM+pK0u+Ad5XZdEZE/LrCbsuAzSPieUmjgRslbU8a\nEpfq11z/XsZbKa5y/7mJbtr3SXexAz8Gzs7nORv4DvCpbmKpFHuj1f3vvA/2ioinJb0TmCLpoW7a\ntvL7gMrxNTvulv7eShoC/Ar4UkT8KU2KlG9aIa5ef77unPooIsb3Yp8V5AcbRsRcSY8D25L+V7Fp\noemmwNN5+RlJG0XEsjxcfrZR8ea4NqsQV7n1y0nD+rfm0VOxfa9VG7ukS4Df5Je1xt5o3cXXVBHx\ndP7zWUk3kKaUKn0PW+V91BrfUmBcyfqpDYgTgIh4pnO51b63klYndUyTIuL6vLphn6+n9ZpA0nBJ\nq+XlLYFtgCfyMPkVSXvm6zbHAJ2jmZuAzkyXYwvrG+Em4AhJa0p6d473PmA2sI1SZt4awBHATfl6\n2e/perhj3eMtub5xCNCZEVVT7PWMsYJWiWMVktaVtF7nMrA/6TOt9D28CTgmZ23tCbzcOf3TYLXG\nNxnYX9I78pTa/nldQ7Tq9zb//vkp8GBEfLewqXGfb39nefhnlYyXQ0j/c1gBPANMzus/ASwiZdrc\nD3y0sM8Y0hf0ceACuqp4bADcATya/1y/UfHmbWfkmB4mZxBGV5bOI3nbGYX1W5L+MT0GXAusWefP\n+iqgA1iQ/6Fs1NvYm/A9aYk4SmLaMn8/5+fv6hndfQ9J0zc/yu+hg0K2Zx1jvIY0Rf56/t5+ujfx\nkabRHss/xzU43pb83gJ7k6bfFgDz8s+Bjfx8Xb7IzMxajqf1zMys5bhzMjOzluPOyczMWo47JzMz\naznunMzMrOW4czIzs5bjzsnMzFrO/wexJpqudqm4fQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10b87fd50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaEAAAEKCAYAAAC7c+rvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmUVeW55/HvUwNVMoOUCFSVEMER\nEbAsSDS3HRN0RdEokaETYpuYlas32p10x5jVbYbb3cnKTXJjpm5zY2tuChAV4xCHoJJoBqooBhlE\npKJyqgAp5lEKqs7Tf5x98FBUUQO1a5/h91nrrNrn3e/e73NQ+bn3eevd5u6IiIhEIS/qAkREJHcp\nhEREJDIKIRERiYxCSEREIqMQEhGRyCiEREQkMgohERGJjEJIREQioxASEZHIFERdQLobNmyYjx49\nOuoyREQyyvLly3e4e0lH/RRCHRg9ejS1tbVRlyEiklHMbFNn+ul2nIiIREYhJCIikVEIiYhIZBRC\nIiISGYWQiIhERiEkIiKRUQiJiEhkFEIiInKCf335barf2Rn6OAohERE5zns7DvKvL2+k5t1doY+l\nEBIRkePMr4mRn2d85tKy0MdSCImIyDFNzS0srK3n2vOHM3xgcejjKYREROSYF9e+z+5DR5kztbxX\nxlMIiYjIMVVLY5x1el8uO3tYr4ynEBIREQDe3rafmvd2MbuynLw865UxFUIiIgLAvOoYffLzuPWS\n0l4bUyEkIiJ8cKSFJ1c0MG38mZzev6jXxlUIiYgIz67ewv7DzcyZ0jsTEpIUQiIiQlV1jLFn9Kdy\nzNBeHVchJCKS49Zu3ssb9XuYXVmOWe9MSEhSCImI5Lh5NTGKCvK4ZXLvTUhIUgiJiOSwA03NPL1y\nMzdcPJJBfQt7fXyFkIhIDvvdys0cPNLS6xMSkhRCIiI5yt2pqo5xwYiBTCwbHEkNCiERkRy1qn4P\n67fuY/aU3p+QkKQQEhHJUVXVMfr1yeemSaMiq0EhJCKSg/YeOsqzb2xh+qRR9C8qiKwOhZCISA56\nckUDTc1xZldGMyEhSSEkIpJj3J15NTEuLhvM+FGDIq1FISQikmNq3t1FXeOByKZlp1IIiYjkmKrq\nGAOKC7hhwsioS1EIiYjkkh0Hmnhh7VZumVzKaX3yoy5HISQikkueWN7A0RZPi1txoBASEckZ8bgz\nvyZG5eihjBs+IOpyAIWQiEjO+Mvfd7Bp5yHmTE2PqyAIMYTMrMzMlpjZejNbZ2b3BO3fMrPNZrYq\neF2fcsw3zKzOzDaY2SdT2qcFbXVmdl9K+xgzqzazjWb2mJn1CdqLgvd1wf7RHY0hIpLtqpbGGNqv\nD9PGnxl1KceEeSXUDHzV3c8HpgJ3mdkFwb4fu/vE4PU8QLBvJnAhMA34hZnlm1k+8HPgOuACYFbK\neb4fnGscsBu4I2i/A9jt7mOBHwf92h0jvD8CEZH0sG3fYRav38aMS0opKkifv/ZCCyF33+ruK4Lt\n/cB64GQLFE0HFrh7k7u/C9QBlcGrzt3fcfcjwAJguiVW27sKeCI4/lHgppRzPRpsPwFcHfRvbwwR\nkay2cFk9LXFnVsQrJLTWK98JBbfDJgHVQdPdZrbazB42syFB2yigPuWwhqCtvfbTgT3u3tyq/bhz\nBfv3Bv3bO1freu80s1ozq92+fXuXP6+ISDppCSYkXD52GKOH9Yu6nOOEHkJm1h94ErjX3fcBvwTO\nBiYCW4EfJru2cbh3o7075zq+wf0hd69w94qSkpI2DhERyRx/3NDIlr2H02ZadqpQQ8jMCkkEUJW7\nLwJw923u3uLuceBXfHg7rAEoSzm8FNhykvYdwGAzK2jVfty5gv2DgF0nOZeISNaqqo5RMqCIay4Y\nHnUpJwhzdpwBvwbWu/uPUtpHpHS7GVgbbD8DzAxmto0BxgE1wDJgXDATrg+JiQXPuLsDS4Bbg+Pn\nAk+nnGtusH0r8GrQv70xRESyUsPuQyzZ0MjMS8sozE+/38oJ8yESlwGfBdaY2aqg7X4Ss9smkrgN\n9h7wJQB3X2dmC4E3Scysu8vdWwDM7G7gJSAfeNjd1wXn+zqwwMz+GVhJIvQIfv67mdWRuAKa2dEY\nIiLZ6LFlia/Bb7u0rIOe0bDEBYK0p6Kiwmtra6MuQ0Sky462xPnY917lolGDePjzl/bq2Ga23N0r\nOuqXftdmIiLSI15+cxvb9zel5YSEJIWQiEiWqqqOMXJQMVece0bUpbRLISQikoXe23GQP9ftYGZl\nOfl5bf12SnpQCImIZKH5NTHy8yxtJyQkKYRERLJMU3MLC2vrufb84QwfWBx1OSelEBIRyTIvrn2f\n3YeOptUjG9qjEBIRyTJV1THOOr0vl509LOpSOqQQEhHJIhu37afm3V3MqiwnL40nJCQphEREskhV\ndYzCfGPGJaVRl9IpCiERkSzxwZEWnlzRwHXjR3B6/6Koy+kUhZCISJZ4dvUW9h9uTusVElpTCImI\nZIl51THGntGfyjFDoy6l0xRCIiJZYO3mvayq38PsynIST9LJDAohEZEsMK8mRlFBHrdMzowJCUkK\nIRGRDHegqZmnV27mhotHMqhvYdTldIlCSEQkwz29ajMHj7Rk1ISEJIWQiEgGc3d+uzTG+SMGMrFs\ncNTldJlCSEQkg62q38P6rfuYMyWzJiQkKYRERDJYVXWMfn3yuWnSqKhL6RaFkIhIhtp76CjPvrGF\n6ZNG0b+oIOpyukUhJCKSoRatbKCpOc7sysybkJCkEBIRyUDuTlV1jIvLBjN+1KCoy+k2hZCISAaq\neXcXdY0HMnJadiqFkIhIBqqqjjGguIAbJoyMupRTohASEckwOw808eLa97llcimn9cmPupxTohAS\nEckwTyxv4EhLPONvxYFCSEQko8TjzryaGJWjhzJu+ICoyzllCiERkQzyl7/vYNPOQ8yZmvlXQaAQ\nEhHJKFVLYwzt14dp48+MupQeoRASEckQ2/YdZvH6bdx6SSlFBZk9ISFJISQikiEWLqunJe7MyuAV\nElpTCImIZICWuDO/JsblY4cxZli/qMvpMQohEZEM8McNjWzZezgrpmWnUgiJiGSAedUxSgYUcc0F\nw6MupUcphERE0lzD7kO8uqGR2yrKKMzPrr+2s+vTiIhkoceW1QMws7Is4kp6XmghZGZlZrbEzNab\n2TozuydoH2pmi81sY/BzSNBuZvagmdWZ2Wozm5xyrrlB/41mNjel/RIzWxMc86AFz7btzhgiIuno\naEucBcvqufLcMygd0jfqcnpcmFdCzcBX3f18YCpwl5ldANwHvOLu44BXgvcA1wHjgtedwC8hESjA\nA8AUoBJ4IBkqQZ87U46bFrR3aQwRkXT18pvb2L6/KesmJCSFFkLuvtXdVwTb+4H1wChgOvBo0O1R\n4KZgezrwG09YCgw2sxHAJ4HF7r7L3XcDi4Fpwb6B7v43d3fgN63O1ZUxRETS0ryaGCMHFXPFuWdE\nXUooeuU7ITMbDUwCqoHh7r4VEkEFJP9kRwH1KYc1BG0na29oo51ujCEiknbe23GQ1zfuYGZlOfl5\nFnU5oQg9hMysP/AkcK+77ztZ1zbavBvtJy2nM8eY2Z1mVmtmtdu3b+/glCIi4ZhfEyM/z7jt0uyb\nkJAUagiZWSGJAKpy90VB87bkLbDgZ2PQ3gCk/kmXAls6aC9to707YxzH3R9y9wp3rygpKen8BxYR\n6SFNzS08vryBa88fzvCBxVGXE5owZ8cZ8Gtgvbv/KGXXM0Byhttc4OmU9s8FM9imAnuDW2kvAZ8w\nsyHBhIRPAC8F+/ab2dRgrM+1OldXxhARSSsvrn2fXQePZM0jG9pTEOK5LwM+C6wxs1VB2/3A94CF\nZnYHEANmBPueB64H6oBDwO0A7r7LzL4LLAv6fcfddwXbXwYeAU4DXghedHUMEZF0U1Udo3xoXy47\ne1jUpYQqtBBy9z/T9ncwAFe30d+Bu9o518PAw2201wLj22jf2dUxRETSxcZt+6l5dxf3XXceeVk6\nISFJKyaIiKSZquoYhfnGjEtKO+6c4RRCIiJp5IMjLTy5ooHrxo/g9P5FUZcTOoWQiEgaeW71FvYf\nbmZ2lq6Q0JpCSEQkjVRVxzi7pB9TxgyNupReoRASEUkTazfvZVX9HuZMOYtgPeaspxASEUkT82pi\nFBXkccvk7J+QkKQQEhFJAweamnl65WZuuHgkg/oWRl1Or1EIiYikgadXbebgkZacmZCQpBASEYmY\nu/PbpTHOHzGQSWWDoy6nVymEREQitqp+D+u37mPOlPKcmZCQpBASEYnYvOoY/frkc9Ok3Hu8WadD\nyMwuN7Pbg+0SMxsTXlkiIrlh76GjPLt6CzdOHEX/ojDXlE5PnQohM3sA+DrwjaCpEPhtWEWJiOSK\nRSsbOHw0zpwcm5CQ1NkroZuBG4GDAO6+BRgQVlEiIrnA3amqjnFx2WDGjxoUdTmR6GwIHQkeg+AA\nZtYvvJJERHJDzbu7qGs8kLNXQdD5EFpoZv8XGGxmXwReBn4VXlkiItlvXk2MAcUF3DBhZNSlRKZT\n34K5+7+Y2bXAPuBc4H+4++JQKxMRyWI7DzTxwpr3mT2lnNP65EddTmQ6FULB7bdX3X2xmZ0LnGtm\nhe5+NNzyRESy0xPLGzjSEs+5FRJa6+ztuNeAIjMbReJW3O3AI2EVJSKSzeJxZ15NjMrRQzlneG7P\n8epsCJm7HwI+DfzU3W8GLgivLBGR7PXXv+9k085DzJma21dB0IUQMrOPAnOA3wdtufdbVSIiPaCq\nehND+hYybfyZUZcSuc6G0D3AfcAid18XrJbwanhliYhkp237DvOHN7cxo6KMooLcnZCQ1NmrmUNA\nHJhlZv8RMILfGRIRkc5buKyelrgzq1K34qDzIVQFfA1YSyKMRESki1rizvyaGJePHcaYYfqdf+h8\nCG1392dDrUREJMv96e1Gtuw9zH//lOZ1JXU2hB4ws38DXgGako3uviiUqkREslDV0hglA4q45oLh\nUZeSNjobQrcD55FYPTt5O84BhZCISCc07D7EqxsaueuKsRTm61FuSZ0NoYvd/aJQKxERyWKPLasH\nYGZlWcSVpJfOxvFSM9NNTBGRbjjaEuexZfVcee4ZlA7pG3U5aaWzV0KXA3PN7F0S3wkZ4O4+IbTK\nRESyxCvrt9G4v4nZmpZ9gs6G0LRQqxARyWJV1TFGDirmyvPOiLqUtNPZRzlsCrsQEZFs9N6Og7y+\ncQf/5dpzyM+zqMtJO5qiISISovk1MfLzjNsu1YSEtiiERERC0tTcwuPLG7j2/OEMH1gcdTlpSSEk\nIhKSF9e+z66DR3L+wXUnoxASEQlJVXWM8qF9uXzssKhLSVuhhZCZPWxmjWa2NqXtW2a22cxWBa/r\nU/Z9w8zqzGyDmX0ypX1a0FZnZveltI8xs2oz22hmj5lZn6C9KHhfF+wf3dEYIiI9beO2/dS8u4vZ\nU8rJ04SEdoV5JfQIbU/t/rG7TwxezwMEvwg7E7gwOOYXZpZvZvnAz4HrSDzJdVbKL81+PzjXOGA3\ncEfQfgew293HAj8O+rU7Rg9/ZhERAObVxCjMN2ZcUhp1KWkttBBy99eAXZ3sPh1Y4O5N7v4uUAdU\nBq86d3/H3Y8AC4DpZmbAVcATwfGPAjelnOvRYPsJ4Oqgf3tjiIj0qA+OtPDk8gamjR/B6f2Loi4n\nrUXxndDdZrY6uF03JGgbBdSn9GkI2tprPx3Y4+7NrdqPO1ewf2/Qv71ziYj0qOdWb2Hf4WbmaEJC\nh3o7hH4JnA1MBLYCPwza27ph6t1o7865TmBmd5pZrZnVbt++va0uIiLtqqqOcXZJP6aMGRp1KWmv\nV0PI3be5e4u7x4Ff8eHtsAYg9Te5SoEtJ2nfAQw2s4JW7cedK9g/iMRtwfbO1VadD7l7hbtXlJSU\ndOejikiOWrdlL6vq9zBnylkkvgmQk+nVEDKzESlvbybxuHCAZ4CZwcy2McA4oAZYBowLZsL1ITGx\n4Bl3d2AJcGtw/Fzg6ZRzzQ22bwVeDfq3N4aISI+ZVx2jqCCPWyZrQkJndHYB0y4zs/nAFcAwM2sA\nHgCuMLOJJG6DvQd8CcDd15nZQuBNoBm4y91bgvPcDbwE5AMPu/u6YIivAwvM7J+BlcCvg/ZfA/9u\nZnUkroBmdjSGiEhPONDUzO9WbuZTE0YyqG9h1OVkBEtcJEh7KioqvLa2NuoyRCQDVFVv4ptPrWXR\nP36MyeVDOj4gi5nZcnev6KifVkwQEekB7k7V0hjnjxjIpLLBUZeTMRRCIiI94I2Gvby5dR9zppRr\nQkIXKIRERHpA1dJN9O2Tz/SJI6MuJaMohERETtHeQ0d5dvUWpk8cxYBiTUjoCoWQiMgpWrSygcNH\n41ohoRsUQiIip8DdqaqOcXHZYMaPGhR1ORlHISQicgqWvbebusYDugrqJoWQiMgpqKrexIDiAm6Y\noAkJ3aEQEhHppp0HmnhhzfvcMrmU0/ro8WTdoRASEemmJ5Y3cKQlzmzdius2hZCISDfE4878mhiV\no4dyzvABUZeTsRRCIiLd8Ne/7+S9nYd0FXSKFEIiIt1QVb2JIX0LmTb+zKhLyWgKIRGRLmrcd5g/\nvLmNGRVlFBdqQsKpUAiJiHTRwtp6WuLOrErdijtVCiERkS5oiTvza+q5fOwwxgzrF3U5GU8hJCLS\nBX96u5HNez7QhIQeohASEemCqqUxSgYUce0Fw6MuJSsohEREOmnzng9YsqGR2yrKKMzXX589QX+K\nIiKd9FhNDAdmVpZFXUrWUAiJiHTC0ZY4C5bVc8U5JZQO6Rt1OVlDISQi0gmvrN9G4/4m5kw5K+pS\nsopCSESkE6qqY4wcVMyV550RdSlZRSEkItKBTTsP8vrGHcysLCc/z6IuJ6sohEREOjCvOkZ+nnHb\npZqQ0NMKoi5ARCRdNbfE+ckrG3no9Xe4/qIRDB9YHHVJWUchJCLShvpdh7hnwUpWxPYw45JSvnXj\nhVGXlJUUQiIirTzzxha+uWgNAA/OmsSNF4+MuKLspRASEQkcbGrmgWfW8cTyBiaXD+YnMydRNlS/\nExQmhZCICLC6YQ/3LFjFpp0H+cpVY/nK1eMo0NI8oVMIiUhOi8edX73+Dv/yhw0M61/E/C9OZcpH\nTo+6rJyhEBKRnNW47zBfffwNXt+4g2kXnsn3brmIwX37RF1WTlEIiUhOevWtbXzt8dUcOtLM//70\nRcy8tAwz/SJqb1MIiUhOOXy0he+98BaP/PU9zh8xkJ/OmsjYMwZEXVbOUgiJSM6oa9zPP81fxfqt\n+/j8x0Zz33XnUVyYH3VZOU0hJCJZz92ZX1PPd55bR78+BTz8+QquOk9PRk0Hoc0/NLOHzazRzNam\ntA01s8VmtjH4OSRoNzN70MzqzGy1mU1OOWZu0H+jmc1Nab/EzNYExzxowc3c7owhItlrz6EjfPm3\nK7j/qTVcOnooL9zzcQVQGglzEvwjwLRWbfcBr7j7OOCV4D3AdcC44HUn8EtIBArwADAFqAQeSIZK\n0OfOlOOmdWcMEcleS9/ZyXU/eZ1X3trG/defx6O3V3KG1n9LK6GFkLu/Buxq1TwdeDTYfhS4KaX9\nN56wFBhsZiOATwKL3X2Xu+8GFgPTgn0D3f1v7u7Ab1qdqytjiEiWaW6J88M/bGD2r5ZSXJjPoi9f\nxp3/cDZ5egxD2unt74SGu/tWAHffambJp0ONAupT+jUEbSdrb2ijvTtjbD3VDyUi6SN14dFbLynl\n2zdeSL8iff2drtLln0xb/3vi3WjvzhgndjS7k8QtO8rLyzs4rYikCy08mnl6e2GkbclbYMHPxqC9\nAUh9WlQpsKWD9tI22rszxgnc/SF3r3D3ipKSki59QBHpfQebmvna42/wlfkrGTe8P8/f83EFUIbo\n7RB6BkjOcJsLPJ3S/rlgBttUYG9wS+0l4BNmNiSYkPAJ4KVg334zmxrMivtcq3N1ZQwRyWBrGvby\nqZ/+mUUrGvjKVWNZ+KWPauXrDBLa7Tgzmw9cAQwzswYSs9y+Byw0szuAGDAj6P48cD1QBxwCbgdw\n911m9l1gWdDvO+6enOzwZRIz8E4DXghedHUMEclM8bjzb39+hx+8pIVHM5klJpdJeyoqKry2tjbq\nMkQkhRYeTX9mttzdKzrqly4TE0REOuXVt7bxXx9fzcEjzfyvmy9iVqUWHs1kCiERyQipC4+ed+YA\nFsyayrjhWng00ymERCTtaeHR7KUQEpG0lbrwaF8tPJqVFEIikpb2HDrCfU+u4cV17/PxccP44YyL\nte5bFlIIiUjaqX5nJ/c+toodB5q4//rz+MLlH9G6b1lKISQiaaO5Jc6Dr2zkZ0vqKB/alye//DEm\nlA6OuiwJkUJIRNJC64VHv3XjhfTXwqNZT/+ERSRyWng0dymERCQyB5uaeeCZdTyxvIHJ5YP5ycxJ\nWvctxyiERCQSaxr28pUFK3lv50H+6aqx3HP1OArye3tNZYmaQkhEelVbC49O1cKjOUshJCK9pnH/\nYb66MLHw6CcvHM73b5mghUdznEJIRHrFkrca+drjb3DwSDP/8+bxzK4s18KjohASkXAdPtrC9198\ni//3Fy08KidSCIlIaLTwqHREISQiPU4Lj0pnKYREpEelLjx6+dhh/OgzWnhU2qcQEpEek1x4dPv+\nJr5x3Xl88eNaeFROTiEkIqes9cKji/5RC49K5yiEROSU1O86xL2PrWL5pt3cMrmUb0/XwqPSefo3\nRUS67dk3tnD/U2vA4SczJzJ94qioS5IMoxASkS5LXXh0UvlgHtTCo9JNCiERaVNL3DnQ1Mz+w0fZ\nf7g5eB1l96Gj/HxJHe/tPMjdV47lnmvGUaiFR6WbFEIiWai5Jc7+w80caGpmX6sQSf25r4225HEH\nmprbPf+ZA4uZ94WpfPRsLTwqp0YhJJJmjjTHT7j62N/UdogkguTEtg+OtnQ4Tp+CPAYWFzCguJAB\nxQUMKC7gjAHFwfaHbQOPbRfSP2gbNfg0rXwgPUIhFJIlGxr59jPryMsz8szIM4KfRn5e4r2lbCf3\n5eWlbBvk51miX7Dv2LZx3LmT/fIM8s1OPHfe8fsSdXDcduq4H5677X7t1dH68xocqysv5XMf16fV\nz5P1SbbR6r2l9D3ZOcN2+GjLCYHw4VXH0WNXGW3vT2w3Ncc7HOe0wvxjIdG/uJCBxQWMHFzMgKLC\nE0JkQLA/ta1/cQFFBQoRiZ5CKCSDTitkQulg4u64J+6vxz35grg7LfHj97XEnSMtKX1S2j15TKvz\nHX9ujo1x3DHJcfzDY3JVW0GVbKPV++ODLQi6vA/DNXmOlrgfC5QjLR0HSL8++ccFwuC+fSgb2vfD\n8CgqaBUkhcddkfQvLtB3MJI1FEIhmVw+hMnlQ6Iuo03eRhAeC7j4h9txd+LxD4MtHk8NwsQ5Wodh\nMiRTx2j9PtkWP65PYts70aftcybfH98nHnecjvvEHRw/bvxj/eMnHp84Z6I936yNq4+CE65KBga3\ns/K1goDIMQqhHJS4lQb5GLqtLyJR0jW9iIhERiEkIiKRUQiJiEhkFEIiIhIZhZCIiERGISQiIpFR\nCImISGQUQiIiEhnzXF7DpRPMbDuwqZuHDwN29GA5mUCfOTfoM+eGU/nMZ7l7SUedFEIhMrNad6+I\nuo7epM+cG/SZc0NvfGbdjhMRkcgohEREJDIKoXA9FHUBEdBnzg36zLkh9M+s74RERCQyuhISEZHI\nKIRCZmY/MLO3zGy1mT1lZoOjrilsZjbDzNaZWdzMsno2kZlNM7MNZlZnZvdFXU/YzOxhM2s0s7VR\n19IbzKzMzJaY2frg3+l7oq4pbGZWbGY1ZvZG8Jm/HeZ4CqHwLQbGu/sE4G3gGxHX0xvWAp8GXou6\nkDCZWT7wc+A64AJglpldEG1VoXsEmBZ1Eb2oGfiqu58PTAXuyoF/xk3AVe5+MTARmGZmU8MaTCEU\nMnf/g7s3B2+XAqVR1tMb3H29u2+Iuo5eUAnUufs77n4EWABMj7imULn7a8CuqOvoLe6+1d1XBNv7\ngfXAqGirCpcnHAjeFgav0CYPKIR6138CXoi6COkxo4D6lPcNZPlfULnMzEYDk4DqaCsJn5nlm9kq\noBFY7O6hfeaCsE6cS8zsZeDMNnZ9092fDvp8k8SlfVVv1haWznzmHGBttGm6aRYys/7Ak8C97r4v\n6nrC5u4twMTgO+ynzGy8u4fyPaBCqAe4+zUn229mc4FPAVd7lsyJ7+gz54gGoCzlfSmwJaJaJCRm\nVkgigKrcfVHU9fQmd99jZn8k8T1gKCGk23EhM7NpwNeBG939UNT1SI9aBowzszFm1geYCTwTcU3S\ng8zMgF8D6939R1HX0xvMrCQ5i9fMTgOuAd4KazyFUPh+BgwAFpvZKjP7P1EXFDYzu9nMGoCPAr83\ns5eirikMwYSTu4GXSHxhvdDd10VbVbjMbD7wN+BcM2swszuirilklwGfBa4K/vtdZWbXR11UyEYA\nS8xsNYn/0Vrs7s+FNZhWTBARkcjoSkhERCKjEBIRkcgohEREJDIKIRERiYxCSEREIqMQEgmJmR3o\nuNdJj3/CzD7SQZ8/drRSeWf6tOpfYmYvdra/yKlQCImkITO7EMh393d6e2x33w5sNbPLentsyT0K\nIZGQWcIPzGytma0xs9uC9jwz+0XwzJbnzOx5M7s1OGwO8HTKOX5pZrUne76LmR0wsx+a2Qoze8XM\nSlJ2zwieEfO2mX086D/azF4P+q8ws4+l9P9dUINIqBRCIuH7NInnslxMYgmUH5jZiKB9NHAR8AUS\nK0wkXQYsT3n/TXevACYA/8HMJrQxTj9ghbtPBv4EPJCyr8DdK4F7U9obgWuD/rcBD6b0rwU+3vWP\nKtI1WsBUJHyXA/ODlYm3mdmfgEuD9sfdPQ68b2ZLUo4ZAWxPef8ZM7uTxH+zI0g8RG91q3HiwGPB\n9m+B1MU2k9vLSQQfJJ4T8zMzmwi0AOek9G8ERnbxc4p0mUJIJHxtPfLhZO0AHwDFAGY2BvgacKm7\n7zazR5L7OpC6JldT8LOFD/+7/8/ANhJXaHnA4ZT+xUENIqHS7TiR8L0G3BY8KKwE+AegBvgzcEvw\n3dBw4IqUY9YDY4PtgcBBYG/LECQ/AAAA9klEQVTQ77p2xskDkt8pzQ7OfzKDgK3BldhngfyUfecQ\n0tL9Iql0JSQSvqdIfN/zBomrk//m7u+b2ZPA1ST+sn+bxBM79wbH/J5EKL3s7m+Y2UpgHfAO8Jd2\nxjkIXGhmy4Pz3NZBXb8AnjSzGcCS4PikK4MaREKlVbRFImRm/d39gJmdTuLq6LIgoE4jEQyXBd8l\ndeZcB9y9fw/V9Row3d1398T5RNqjKyGRaD0XPECsD/Bdd38fwN0/MLMHgFFArDcLCm4Z/kgBJL1B\nV0IiIhIZTUwQEZHIKIRERCQyCiEREYmMQkhERCKjEBIRkcgohEREJDL/Hy8jKKtlNBs/AAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x110f7fb50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot important coefficients\n",
    "coefs = pd.Series(ridge.coef_, index = feat_names)\n",
    "print(\"Ridge picked \" + str(sum(coefs != 0)) + \" features and eliminated the other \" +  \\\n",
    "      str(sum(coefs == 0)) + \" features\")\n",
    "\n",
    "#正系数值最大的10个特征和负系数值最小（绝对值大）的10个特征\n",
    "imp_coefs = pd.concat([coefs.sort_values().head(10),\n",
    "                     coefs.sort_values().tail(10)])\n",
    "imp_coefs.plot(kind = \"barh\")\n",
    "plt.title(\"Coefficients in the Ridge Model\")\n",
    "plt.show()\n",
    "\n",
    "mse_mean = np.mean(ridge.cv_values_, axis = 0)\n",
    "plt.plot(np.log10(alphas), mse_mean.reshape(len(alphas),1)) \n",
    "\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "season_1        -816.946054\n",
       "season_2          61.439057\n",
       "season_3          97.124573\n",
       "season_4         658.382424\n",
       "mnth_1          -384.604861\n",
       "mnth_2          -255.398769\n",
       "mnth_3           185.553801\n",
       "mnth_4            -6.012570\n",
       "mnth_5           437.523845\n",
       "mnth_6           104.170972\n",
       "mnth_7          -307.981162\n",
       "mnth_8           129.351248\n",
       "mnth_9           670.800620\n",
       "mnth_10          235.474439\n",
       "mnth_11         -428.094512\n",
       "mnth_12         -380.783051\n",
       "weathersit_1     757.284463\n",
       "weathersit_2     341.945472\n",
       "weathersit_3   -1099.229935\n",
       "weekday_0       -166.919676\n",
       "weekday_1       -155.850725\n",
       "weekday_2        -48.116950\n",
       "weekday_3         -4.759450\n",
       "weekday_4         64.295894\n",
       "weekday_5         87.446625\n",
       "weekday_6        223.904283\n",
       "temp            1892.310062\n",
       "atemp           1605.349484\n",
       "hum            -1520.738791\n",
       "windspeed      -1336.567102\n",
       "holiday         -202.590794\n",
       "workingday       145.606188\n",
       "yr              1952.328786\n",
       "dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coefs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "相比OLS，岭回归模型增加了L2正则，系数值进行了收缩。\n",
    "由于增加正则限制了模型复杂的，相比比OLS模型，岭回归模型在训练集上的误差略有增大，但在测试集上的误差有所减小。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "2da33461-fb1a-c09a-b0fe-d20e8c8f52e3",
    "_uuid": "d2e8bab153344d6ecf2e48909b76bd8f387e2710"
   },
   "source": [
    "**3* Linear Regression with Lasso regularization (L1 penalty)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "_cell_guid": "8525724a-fb77-66d4-e06f-f3365fbd8ec3",
    "_uuid": "f8f41944e405693ba867158353ffedcd9ecd4a07",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('Best alpha :', 2.336453166053944)\n",
      "('cv of rmse :', 828.55504605086469)\n",
      "Lasso picked 27 features and eliminated the other 6 features\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAacAAAEICAYAAAD7pTujAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmYXFWd//H3R3ZIJCwRQZbIZobN\nQJptIE6ACIoLOoaHbZQgI4PIMP4cWQYYjCIjGAURFwSMbBlEEDQIGiIYEkIIJJBVtkBgiCAkSFgU\nIpDv749zitw0Vd3V3dW1dH9ez1NPqs49995zb6fr2+fcc79XEYGZmVkzeVejG2BmZtaeg5OZmTUd\nByczM2s6Dk5mZtZ0HJzMzKzpODiZmVnTcXCylibpA5IelPSKpFMkrSfpFkkvSbpB0jGSbq9iO2dK\nuqIebe6gDVtLelXSGjXa3lhJ19ZiW/2NpCcljaqi3hBJIWnNerSrP3FwsrqQdLSkWfnL91lJv5W0\nfw02fRowJSIGRsT3gdHAZsAmEXF4REyIiIM720hE/E9E/GtPG9OTL6uI+L+IGBARb3VjvyMlLenq\nel3Y/pWSvtlb2++u3K6Q9Ml25d/L5WMa1DTrIQcn63WSvgJ8D/gfUuDYGvgRcFgNNr8NsLDd50cj\n4s0abNtaw6PAsaUP+Q+Dw4HHG9Yi6zEHJ+tVkjYEvgF8KSJuioi/RsQbEXFLRJya66yT/9J9Jr++\nJ2mdwjY+LmmOpOWS7pG0Wy6/EzgA+EHukV0HnAMckT8fL2mMpLsL29pZ0mRJf5H0nKQzc/lqQ2CS\n9sn7Wi5prqSRhWVTJJ0raXoeTrxd0qZ58dT87/Lchn0lbS/prjzUuEzS9RXO1Wq9rk72U1xvA+C3\nwBZ5n69K2iIvXlvS1Xn9hZLaCuttIemXkpZKWizplGp+pmX2f7GkpyW9LGm2pBGFZXvlHvPL+Xxf\nmMvXlXStpBfyOb5f0maFdk3MP6NFkr7QSRNuAfaTtFH+/BFgHvDnQjveJelsSU9Jej6fkw0Lyz+b\nl70g6ax2x/cuSWdIejwv/4Wkjbtzrqx6Dk7W2/YF1gVu7qDOWcA+wDDgg8BewNkAkvYAxgP/BmwC\n/ASYKGmdiDgQmAacnIfDjiL1zq7Pn39a3ImkgcDvgd8BWwDbA3e0b4yk9wG3At8ENga+CvxS0uBC\ntaOB44D3AGvnOgAfyv8Oym2YAZwL3A5sBGwJXNLBuWiv0n7eFhF/BT4KPJP3OSAinsmLPwn8HBgE\nTAR+kI/xXaQv9bnA+4CDgC9LOqQLbSu5n/Sz2xj4X+AGSevmZRcDF0fEu4HtgF/k8mOBDYGtSD/X\nE4HX8rLrgCWkn9Fo4H8kHdTB/l/Px3Zk/vw54Op2dcbk1wHAtsAAVp2LnYAfA5/N+9yE9HMqOQX4\nFPBPefmLwA87aI/VgIOT9bZNgGWdDLMdA3wjIp6PiKXA10lfFABfAH4SETMj4q2IuApYQQpmXfVx\n4M8R8d2IeD0iXomImWXq/QtwW0TcFhErI2IyMAs4tFDnZxHxaES8RvrCHdbBft8gDTdukfd7dwd1\n2+vKfsq5Ox/HW8A1pOAPsCcwOCK+ERF/j4gngMtZ9QVftYi4NiJeiIg3I+K7wDrAB/LiN4DtJW0a\nEa9GxL2F8k2A7fPPdXZEvCxpK2B/4PR8ruYAV7Dq/0MlVwOfy72hfwJ+1W75McCFEfFERLwK/Bdw\nZO6ljgZ+ExFTI2IF8N/AysK6/wacFRFL8vKxwGh5EkSvcnCy3vYCsGknv8hbAE8VPj+VyyB9qf9n\nHvpZLmk56a/tLei6rajuOsQ2wOHt9rk/sHmhzp8L7/9G+ku8ktMAAfflobXPd6HNXdlPNeuvm38W\n25CGAYvHeCbpmmCXSPpPSQ/lYcvlpB5RafjxeGBH4OE8dPfxXH4NMAn4udJQ7rclrUX6uf4lIl4p\n7OIpUu+uohzwB5N63L/Jwbyo3P+xNfPxbgE8XdjWX0n/b0u2AW4unKeHgLfoxrmy6jnyW2+bQRp2\n+RRwY4U6z7D6xIatcxmkL43zIuK8GrTlaeCoKutdExGdXeso5x1p/iPiz6QeIEozFH8vaWpELOrG\n9qvebyeeBhZHxA492Wm+vnQ6aVhwYUSslPQiKRgTEY8BR+VhxH8GbpS0SQ4AXwe+LmkIcBvwCGn4\nc2NJAwsBamvgT1U051rSNccDyiwr/R8r2Rp4E3gOeBb4h8IxrU/q1ZU8DXw+IqaXOf4hVbTLusE9\nJ+tVEfES6Qvjh5I+JWl9SWtJ+qikb+dq1wFnSxqcL/ifQ/qigTTUdKKkvZVsIOlj+fpRV/0GeK+k\nLytNwhgoae8y9a4FPiHpEElr5Iv3IyVtWaZue0tJQ0LblgokHV5Y90VSIOnydPFOPAdsUrzI34n7\ngJclna50b9gaknaRtGcH65TORem1NjCQ9CW/FFhT0jnAu0srSPoXSYMjYiWwPBe/JekASbsq3dP1\nMmmY762IeBq4B/hW3sdupN7XhCqO6fvAh1k1KaXoOuD/SXq/pAGsujb5JumPpo9L2j8f0zdY/bvx\nUuA8SdvkYxosqRYzTa0DDk7W6yLiQuArpCGXpaS/RE9m1XWBb5Ku6cwD5gMP5DIiYhap1/ED0hf7\nItKF7e604xXSl9cnSMNdj1Hmr+z8BXkYaZir1N5TqeL3JSL+BpwHTM/DQPuQru/MlPQq6cL9f0TE\n4u4cQwf7fZj0BfxE3m+Hw575GtQnSNewFgPLSNd2OgpuZ5AmLZRed5KG5n5Lms79FKmX/HRhnY8A\nC/OxXwwcGRGvA+8lBYWXScNkd7HqD5KjgCGk3s7NwNfydb/OzsFfIuKOKP+QuvGkocSp+XhfB/49\nr7cQ+BJpMsezpP9nxXvGLib93G6X9ApwL1DujxqrIflhg2Zm1mzcczIzs6bj4GRmZk3HwcnMzJqO\ng5OZmTUd3+fUTZtuumkMGTKk0c0wM2sps2fPXhYRgzur5+DUTUOGDGHWrFmNboaZWUuR9FTntTys\nZ2ZmTahle06SBgFHR8SPGt0W69uGnHFro5tg1lSePP9jvb6PVu45DQJOanQjzMys9lo5OJ0PbKf0\nELpxkk7NWY/nSfo6vP3wtoclXSFpgaQJkkYpPbztMUl75XpjJV0j6c5c3p2En2ZmViOtHJzOAB6P\niGHAZGAH0kPqhgHDJZUe+rY9KTfWbsBQ0sPb9ic9tO3MwvZ2Az5GejjeOeVyk0k6QempnrOWLl3a\nO0dlZmYtHZyKDs6vB0lJQ4eSghWkxwLMz1mRFwKlxJDzScklS34dEa9FxDLgD6RAt5qIuCwi2iKi\nbfDgTmdCmplZN7XshIh2BHwrIn6yWmF61sqKQtHKwueVrH787TPgOiOumVmDtHJweoX0LBlIafvP\nlTQhIl6V9D7S82G64jBJ3wI2AEaShg3N6jIzycxW17LBKSJeyBMbFpCeJ/O/wAxJAK8C/0LXHuh2\nH3Ar6QmZ50bEM53UNzOzXtKywQkgIo5uV3RxmWq7FOqPKbx/srgMeDQiTqhl+8zMrHv6yoQIMzPr\nQ1q651QrETG20W0wM7NVHJzMOuH0ReZJMfXnYb0KJK3R6DaYmfVX/bbnJOlcYFlEXJw/nwc8B3wa\neJaUaWKnxrXQzKz/6s89p58CxwJIehdwJPAnUmaIsyLiHYHJ6YvMzOqj3wanPJX8BUm7syr10QvA\nfRGxuMI6Tl9kZlYH/XZYL7sCGAO8Fxify/7asNZYU/LFcLP667c9p+xm4CPAnqQUSGZm1gT6dc8p\nIv4u6Q/A8oh4K6c+MjOzBuvXwSlPhNgHOBwgIqYAUxrYJDMzox8P60naCVhEer7TY41uj5mZrdKy\nwUnSmZ3Xqiwi/hgR20bEf9aqTWZmVhutPKx3JvA/jW6E9X39JX2RZyVaM2mJnpOkX0maLWlhvhH2\nfGA9SXMkTch1/kXSfbnsJ6X0Q5JelXRBXv/3kvaSNEXSE5I+meuMkfRrSb+T9IikrzXwcM3M+r2W\nCE7A5yNiONAGnAKMA16LiGERcYykfwCOAPaLiGGkhwwek9fdAJiS138F+CbwYVKaom8U9rFXXmcY\ncLiktjocl5mZldEqw3qnSPp0fr8VsEO75QcBw4H783Tw9YDn87K/A7/L7+cDKyLiDUnzgSGFbUyO\niBcAJN0E7A/MKu5E0gnACQBbb711z4/KzMzKavrgJGkkMArYNyL+JmkKsG77asBVEfFfZTbxRkRE\nfr8SWAEQESslFY8/2q3X/jMRcRlwGUBbW9s7lpuZWW00fXACNgRezIFpKOm+JIA3JK0VEW8AdwC/\nlnRRRDwvaWNgYEQ81YX9fDiv9xrwKeDztTwIa12eKGBWf61wzel3wJqS5gHnAvfm8suAeZImRMQf\ngbOB23O9ycDmXdzP3cA1wBzglxExq5P6ZmbWS7RqxKv/kjQGaIuIk6tdp62tLWbNcvwyM+sKSbMj\notMJZ63QczIzs36mFa459bqIuBK4ssHNMDOzzD0nMzNrOn2m5yRpEHB0RPwofx4JfDUiPl7l+gcC\n3wHWBmYDx0fEm73UXGshrZS+yDMLra/oSz2nQcBJ3VkxPzrjKuDIiNgFeAo4toZtMzOzLmiq4CRp\niKSHJV0haYGkCZJGSZou6bGcF2+spPGF/Hin5NXPB7bLufXG5bIBkm7M25ygyk8T3ISUOeLR/Hky\n8Jky7TtB0ixJs5YuXVrTYzczs1WaKjhl2wMXA7sBQ4GjSamEvkrKRE4uP4SUD+9rktYCzgAez/n2\nTs31dge+DOwEbAvsV2Gfy4C1Cvn0RpPSJK0mIi6LiLaIaBs8eHDPjtLMzCpqxuC0OCLmR8RKYCHp\nYYBByos3JNe5NSJWRMQyUg69zSps676IWJK3NYfVc+m9LW//SOAiSfeREsT6epOZWYM044SIFYX3\nKwufV7KqvcU6b1H5OKqtR0TMAEYASDoY2LH6Jltf5kkGZvXXjD2n7noFGNjdlSW9J/+7DnA6cGmN\n2mVmZl3UZ4JTftzF9DyRYlynK7zTqZIeAuYBt0TEnbVtoZmZVcu59brJufXMzLrOufXMzKxlNeOE\niF4l6Wbg/e2KT4+ISY1oj5mZvVO/C04R8emOlkuaCGybM0WYNSx9kWcJWn/mYb0CSf8MvNrodpiZ\n9Xd1C06SNpB0q6S5eUbdEZKGS7pL0mxJkyRtnut+QdL9ue4vJa2fyw/P686VNDWXrSvpZ5LmS3pQ\n0gG5fIykmyT9Lqc++nYn7RsAfAX4Zgd1nL7IzKwO6tlz+gjwTER8MA+Z/Q64BBgdEcOB8cB5ue5N\nEbFnRHwQeAg4PpefAxySyz+Zy74EEBG7AkcBV0laNy8bBhwB7AocIekdKYkKzgW+C/ytUgWnLzIz\nq496Bqf5wChJF0gaQcpdtwswWdIc4Gxgy1x3F0nTJM0HjgF2zuXTgSslfQFYI5ftD1wDEBEPkzKK\nl7I73BERL0XE68AfgW3KNUzSMGD7iLi5dodrZmbdVbcJERHxqKThwKHAt0iZvxdGxL5lql8JfCoi\n5koaA4zM2zhR0t7Ax4A5OahUyjQO1acv2hcYLunJXOc9kqZExMjqjs76Mk9MMKu/el5z2gL4W0Rc\nS3qo397AYEn75uVrSSr1kAYCz+Zs48cUtrFdRMyMiHNImcS3AqaW6kjaEdgaeKQrbYuIH0fEFhEx\nhNQTe9SBycysceo5lXxXYJyklcAbwBdJmb+/L2nD3JbvkTKR/zcwkzREN59VOfPGSdqB1Fu6A5gL\nPAxcmocA3wTGRMSKyo9uMjOzZuf0Rd3k9EVmZl3n9EVmZtay+l2GCEkzgXXaFX82IuY3oj3W/OqV\nIcITL8xWqXvPKd8cu0Xh85OSNu2F/dwmaVB+nVQqj4i986Pci6/5+Wbd5ZJ+U+u2mJlZ1zRiWG8M\nsEVnlaohqaMn2x4aEcuBQcBJleoVjAM+W4t2mZlZz3QanCSdJumU/P4iSXfm9wdJulbSwZJmSHpA\n0g05DRCSzskpiBZIukzJaKANmCBpjqT18m7+Pa8/X9LQvP4GksbnbTwo6bBcPibv5xbgdkmbS5qa\nt7cg3+Bb7JGdD2yXl1d8CGFE3EF6mm5H58Lpi8zM6qCantNUYER+3wYMyPcf7U+a5n02MCoi9gBm\nkfLTAfwgpyDaBVgP+HhE3JjrHJOH017LdZfl9X8MfDWXnQXcGRF7AgeQppFvkJftCxwbEQcCRwOT\nImIY8EFgTrv2nwE8nvd3ajUnpRKnLzIzq49qJkTMJmVPGEjKuPAAKUiNACYCO5Eejw6wNjAjr3eA\npNOA9YGNSfcv3VJhHzcV9vXP+f3BwCcllYLVuqQbbAEmR8Rf8vv7gfE5YP4qItoHJzMzazGdBqeI\neCOn9TkOuAeYR+rJbAcsJgWKo4rr5MSrPwLaIuJpSWNJwaWSUpqhYoohAZ+JiNWyPeT0RX8ttG+q\npA+RUhpdI2lcRFzd2XGZVcuz6Mzqr9oJEVNJw21TgWnAiaThs3uB/SRtDyBp/ZxCqBSIluVrUKML\n23qFVRkfOjKJdC1Kedu7l6skaRvg+Yi4HPgpsEe7KtXuz8zMmkS1wWkasDkwIyKeA14HpkXEUtLs\nu+skzSMFq6F5ltzlpGtSvyINvZVcSUo3VJwQUc65wFrAPEkL8udyRpKSwD4IfAa4uLgwIl4gDTsu\n6GhChKRpwA3AQZKWSDqkg7aZmVkvcvqibnL6IjOzrnP6IjMza1n9Kn2RpF3JDyYsWBERezeiPdYa\nejt9kSdcmL1Tn+k5tU9TJGlk+1REETG/TOqivXP9KyUtztfCSg8yNDOzBugzwYnq0xR15NRC0PL9\nUmZmDdJUwUnSEEkPS7oiz66bIGmUpOmSHpO0l6SxOa3RFElPlFIrUT5N0QBJN+ZtTihNS+9B+5y+\nyMysDpoqOGXbk6aD7wYMJaUn2p90n9WZuc5Q4BBgL+BrOTtEuTRFuwNfJmWx2BbYr5N9nydpXs4h\n2P6xGk5fZGZWJ80YnBbna0MrSSmP7og0330+MCTXuTUiVkTEMuB5YLMK27ovIpbkbc0prF/Of5GC\n3p6kdEun9/hIzMysW5pxtt6KwvuVhc8rWdXeYp1iyqOOttVRPSLi2dI6kn7GqgS01s95Np1Z/TVj\nz6m7epSmSNLm+V8BnwIW1KhdZmbWRc3Yc+qWiHghT5xYAPwW6OrNKRMkDSYlnJ1Dyh9oZmYN4PRF\n3eT0RWZmXef0RWZm1rL6zLBetSTdDLy/XfHpETGpEe2x5uf0RWb111I9p2pSFHWy/snArqTHuY8q\nZYMAbpf0fUmL8n1O7Z8JZWZmddRSwYmepyiaDowCnmpX/lFgh/w6AfhxD/ZhZmY9VPfg1MgURRHx\nYEQ8WWbRYcDVkdwLDCpNLW/XdqcvMjOrg0b1nBqZoqic9wFPFz4vyWWrcfoiM7P6aFRwalSKokrK\n9bY8x97MrEEaNVuvISmKOrAE2KrweUvgmW5sx/ogz6Yzq79WmxDRoxRFHZgIfE7JPsBLhVx7ZmZW\nZy0VnCLiBWB6nkgxrtMV2pF0iqQlpJ7RPElX5EW3AU8Ai4DL6flDC83MrAecvqibnL7IzKzrnL7I\nzMxaVp9MX+QURVZLtUpf5IkVZtXrMz2nYmqjiPg06d6nJaUURZ0FJkk/lTQ3py+6UdKAerTbzMze\nqc8EJ3qe2uj/RcQHI2I34P+Ak2vTLDMz66qmCk4NTm30cm6DgPUocxOu0xeZmdVHUwWnrGGpjST9\nDPhz3v4l7Zc7fZGZWX0044SIxRExH0DS26mNJJVSG80hpzYCVkjqNLVR3lYptdHdlXYcEcdJWoMU\nmI4AflabQ7JW5okMZvXXjD2nhqY2ioi3gOuBz1TTWDMzq71mDE7d1e3URjlt0fal98AngIdr2DYz\nM+uCZhzW65aIeCFPnFgA/Bboys0pAq6S9O78fi7wxV5oppmZVcHpi7rJ6YvMzLrO6YvMzKxlNd2w\nnqQpwFcjomK3RNIYoC0iunyjbKXURsCGwFjS/U1zI+Lorm7b+qZK6Ys8i8+s9zRdcOptObXRaiTt\nAPwC2C8iXpT0nvq3zMzMSno8rCfptFKWBkkXSbozvz9I0rWSDpY0Q9IDkm4o5ayTNFzSXZJmS5ok\nafN2232XpKskfTN/Pk7So5LuonAzraRPSJop6UFJv5e0WV73MUmDC9taJGnTCofxBeCHEfEiQEQ8\n39PzYmZm3VeLa05TgRH5fRspZdBapKwO84GzgVERsQcwC/hKXn4JMDoihgPjgfMK21wTmAA8GhFn\n58D1dVJQ+jAp40PJ3cA+EbE78HPgtIhYCVwLHJPrjCIN1S2rcAw7Ajvm2X73SvpIuUpOX2RmVh+1\nGNabDQyXNJB00+sDpCA1gvT4851IT68FWBuYAXwA2AWYnMvXAIqPRf8J8IuIKAWsvYEpEbEUQNL1\npIAC6am21+cAtjawOJePB34NfA/4PB1ne1gT2AEYmbc3TdIuEbG8WCkiLgMugzRbr/NTY2Zm3dHj\n4BQRb0h6EjgOuAeYBxwAbEcKFJMj4qjiOpJ2BRZGxL4VNnsPcICk70bE66VdVah7CXBhREyUNJI0\nqYGIeFrSc5IOJAW3YyqsD7AEuDci3gAWS3qEFKzu72Ad6yc88cGs/mo1lXwqKTHrVGAacCIpB969\nwH6F7AvrS9oReAQYLGnfXL6WpJ0L2/spcBtwg6Q1gZnASEmb5CHBwwt1NwT+lN8f265dV5CG936R\n0xJV8itSQCVfl9oReKILx29mZjVUq+A0DdgcmBERzwGvA9PyMNwY4DpJ80jBamhE/B0YDVwgaS4p\nkP1jcYMRcSFpiPAa4DlSj2gG8PtcXjKWFMSmAe2vKU0EBtB5AtdJwAuS/gj8ATg1Il6o9uDNzKy2\n+nSGCEltwEURMaLTyl3kDBFmZl1XbYaIPnufk6QzSPnxOrrWZGZmTajPBqeIOJ/0dNy3STqL1a9X\nAdxQmBVoZmZNoM8EJ0mDgKMj4kf580hSGqSPl+rkIFQ2EEk6mfTU3O2AwR3cE2V9RKW0RO15tp5Z\n/fWlxK+DgJN6sP500s26T9WmOWZm1l1NFZwkDZH0sKQrJC2QNEHSqJy54TFJe0kaK2m8pCmSniil\nTiIN4W0naY6kcblsgKQb8zYn5AcJlhURD0bEk719jGZm1rlmHNbbnnRd6ATSTbBHk1IhfRI4kzTt\nfCjpvqSBwCOSfgycAewSEcPg7WG93YGdgWdIPaP9SOmOukXSCbldbL311t3djJmZdaKpek7Z4oiY\nn/PjLQTuiDTffT4wJNe5NSJW5OtCzwObVdjWfRGxJG9rTmH9bomIyyKiLSLaBg8e3JNNmZlZB5qx\n57Si8H5l4fNKVrW3WOctKh9HtfWsH/JEB7Pm1Yw9p+56hTTMZ2ZmLa7PBKecbmh6nkgxrtMV2pF0\niqQlpKzk8yRdUfNGmplZVfp0+qLe5PRFZmZdV236oj7TczIzs76j300QkHQz8P52xadHxKRGtMfM\nzN6ppYJTNSmKOln/ZGBX2qUoknRMfqQHwKvAFyNibq3bb83F6YvMmlerDev1VoqixcA/RcRuwLnk\nR7GbmVlj1D04NWOKooi4JyJezB/vJc3YK9f2EyTNkjRr6dKl3T8JZmbWoUb1nLYHLgZ2I6UiKqUo\n+iopRRG5/BBgL+Br+fHsZwCPR8SwiDg119udlE18J2BbUoqinjge+G25Bc4QYWZWH40KTk2ZokjS\nAaTgdHp3t2FmZj3XqAkRTZeiSNJuwBXAR/MNvdbHeaKDWfNqtQkRvZKiSNLWwE3AZyPi0Vpv38zM\nuqalglMvpig6B9gE+FGebOHUD2ZmDeT0Rd3k9EVmZl3n9EVmZtayWipDRLWcosjMrLW1VHCqNn1R\nRHy6wvonk+6Jap++aCjwM2AP4KyI+E6vHYQ1TLXpitrzrD6z+mu1Yb3eSl/0F+AUwEHJzKwJOH1R\nKn8+Iu4H3uik7U5fZGZWB05f1AVOX2RmVh9OX2RmZk3H6Yus3/DEBrPW0WoTInolfZGZmTWXlgpO\nvZW+SNJ7c/lXgLMlLZH07po23szMqub0Rd3k9EVmZl3n9EVmZtay+uTkAacvMuh+Roj2PJHCrP6a\nLjhJmkJKSVRxzEzSGKAtIk4ut7xS+qIOtrcOcDUwHHgBOKLczbpmZlYfHtZLjgdejIjtgYuACxrc\nHjOzfq3HwUnSaaX0QpIuknRnfn+QpGslHSxphqQHJN0gaUBePlzSXZJmS5okafN2232XpKskfTN/\nPk7So5LuopAFQtInJM2U9KCk30vaLK/7mKTBhW0tkrRphcM4DLgqv78ROKhcGiSnLzIzq49a9Jym\nAiPy+zZSrru1SOmI5gNnA6MiYg9gFvCVvPwSYHREDAfGA+cVtrkmMAF4NCLOzoHr66Sg9GFSqqKS\nu4F9ImJ34OfAaTlbxLXAMbnOKGBuKQt5Ge8DngaIiDeBl0hPxl2N0xeZmdVHLa45zQaGSxpIytbw\nAClIjQAmkgLJ9NwRWRuYAXwA2AWYnMvXAJ4tbPMnwC8iohSw9gamRMRSAEnXAzvmZVsC1+cAtjaw\nOJePB34NfA/4POmRGJWUSxbrOfZmZg3S4+AUEW9IehI4DrgHmAccQHpm0mJgckQcVVxH0q7AwojY\nt8Jm7wEOkPTdiHi9tKsKdS8BLoyIifn5TmNzu56W9JykA0nB7ZgK6wMsAbYClkhaE9iQ9BgNa2Ge\nZWfWumo1IWIqKaP4VGAacCIpCeu9wH6StgeQtL6kHYFHgMGS9s3la0naubC9nwK3ATfkYDETGClp\nkzwkeHih7obAn/L7Y9u16wrS8N4vIuKtDto/sbDuaODO8N3JZmYNU6vgNA3YHJgREc8BrwPT8jDc\nGOA6SfNIwWpoRPydFAQukDSXFMj+sbjBiLiQNER4DfAcqUc0A/h9Li8ZSwpi04D215QmAgPoeEgP\nUjDcRNIiUgqjM6o9cDMzq70+nb5IUhtwUUSM6LRyFzl9kZlZ11WbvqjpbsKtFUlnAF+k42tNZmbW\nhJouOEkaAvwmInapsv6Vuf6NOcv4hRHxx4g4n/RY97czSpBmBB7ebhM3FGYFWh9Qq7RFJZ5YYVZ/\nTReceiIi/rWT5eex+v1UZmZ7epNAAAAPZElEQVTWhJo1fdEaki6XtFDS7ZLWkzRM0r2S5km6WdJG\n7VeSNCVfZ+rtjBJmZtaLmjU47QD8MCJ2BpYDnyElZj09InYjZZ74WqWVeyujhNMXmZnVR7MGp8UR\nMSe/n026oXdQRNyVy64CPtTB+m9nlMjT1q8vLNsSmCRpPnAqULq/ajzwufy+bEYJpy8yM6uPZg1O\nKwrv3wIGdWMbHWWU+EFE7Ar8G7AupIwSQDGjxG+7sU8zM6uBVpkQ8RLwoqQRETEN+CxwVwf1ZwIX\nS9oEeJk0Q29uXlZNRolrOskoYU3Ms+vMWl+rBCdIgeRSSesDT5By+ZUVEc9KGkvKKPEsKaPEGnnx\nWFJGiT+RMlYUn5g7kTSc11lGCTMz60V9OkNEV3Ulo4QzRJiZdV2/zxDRVc4oYWbWPJp1QkTdRcT5\nEbFNRNzd6LaYmfV37jlZy6p1mqJKPMHCrP76TM9J0iBJJxU+j5T0my6sP03SnPx6RtKveqelZmbW\nmT4TnEj3Qp3Uaa0KImJERAyLiGGkWX431axlZmbWJU0VnCQNkfSwpCskLZA0QdIoSdNz7ru9JI2V\nND7n0XtC0il59fOB7XLPZ1wuGyDpxrzNCZJURRsGAgcC7+g5OX2RmVl9NFVwyrYHLgZ2A4YCRwP7\nkx4Df2auMxQ4BNgL+Fp+dPsZwOO593Nqrrc78GVSbr1tKSSA7cCngTsi4uX2C5y+yMysPpoxOC2O\niPk5GetCUqAIUrLXIbnOrRGxIidmfR7YrMK27ouIJXlbcwrrd+Qo4LqeHICZmfVMM87WK+bVW1n4\nvJJV7W2fe6/ScVRbD4Cc7mgvUu/Jmpxn0Zn1Xc3Yc+quV4CBPdzG4aSn6r5eg/aYmVk39ZngFBEv\nANPzRIpxna5Q3pF4SM/MrOGcW6+bnFvPzKzrqs2t12d6TmZm1nc044SIXiXpZlZ/TAbA6aQn634O\n2CgiBtS9YU2oXumBmp0nXpjVX78LThFRdiaepJeAHwCP1bdFZmbWXt2G9SRtIOlWSXPzpIUjJA2X\ndJek2ZImSdo81/2CpPtz3V/mBwwi6fC87lxJU3PZupJ+Jmm+pAclHZDLx0i6SdLvcnaJb3fUvoi4\nNyKe7e3zYGZmnavnNaePAM9ExAcjYhfgd8AlwOiIGA6MB87LdW+KiD0j4oPAQ8Dxufwc4JBc/slc\n9iWAiNiVdAPtVZLWzcuGAUcAuwJHSNqqJwfg9EVmZvVRz+A0Hxgl6QJJI4CtgF2AyZLmAGcDW+a6\nu+Qs4fNJD//bOZdPB66U9AVWPXZ9f+AagIh4GHgK2DEvuyMiXsr3Lf0R2KYnB+D0RWZm9VG3a04R\n8aik4cChwLeAycDCiNi3TPUrgU9FxFxJY4CReRsnStob+BgwR9IwoKNkrl3KEGGr80QAM2uUel5z\n2gL4W0RcC3wH2BsYLGnfvHwtSaUe0kDg2ZzQ9ZjCNraLiJkRcQ6wjNT7mlqqI2lHYGvgkTodlpmZ\n9YJ69iR2BcZJWgm8AXwReBP4vqQNc1u+R0r2+t/ATNIQ3XxWpSUaJ2kHUm/pDmAu8DBwaR4CfBMY\nExErqng6xmryhImjgfUlLQGuiIix3T9cMzPrLmeI6CZniDAz6zpniDAzs5ZVz2tOt0ka1IX6QyQt\n6IV2zMxPyy2+dm1X59Va79fMzKpXz9l6h9ZrXx2JiL0b3Yaeclqh+vKsRbP6q1nPSdJpkk7J7y+S\ndGd+f5CkayU9KWnT3CN6SNLlkhZKul3Sernu8Jz9YQb55tpcvrOk+3IvZ56kHfJ2HpZ0VS67sZBJ\nolLmie1yxojZ+T6qobn8/ZJm5KwU59bqnJiZWffUclhvKjAiv28DBuSp4PsD09rV3QH4YUTsDCwH\nPpPLfwacUubepxOBiyNiWN72klz+AeCyiNgNeBk4Ke+zUuaJy4B/z+VfBX6Uyy8GfhwRewJ/7u4J\nMDOz2qhlcJoNDJc0kHTz6wxSIBnBO4PT4oiYU1hvSJ5OPigi7srl1xTqzwDOlHQ6sE1EvJbLn46I\n6fn9taRA+AHKZJ6QNAD4R+CGXP4TYPO87n6seshgcb+rcfoiM7P6qNk1p4h4Q9KTwHHAPcA84ABg\nO1J+vKL2mRvWI927VHZee0T8r6SZpMwQkyT9K/BEmfqRt/OOzBOS3g0sz72vsrvp8ABTOy4j9b5o\na2vzHHwzs15S6wkRU0nDZZ8n3Tx7ITA7IqKzm2IjYrmklyTtHxF3s3pmiG2BJyLi+/n9bqTgtLWk\nfSNiBinp692k7BCDS+V5mG/HiFgoabGkwyPiBqUG7RYRc0k5+44k9b6Oocn5Ar2Z9XW1nko+jTRU\nNiMingNe551Deh05DvhhnhDxWqH8CGBBHo4bClydyx8CjpU0D9iYdN3o78Bo4AJJc4E5pOE8SIHn\n+Fy+EDgsl/8H8CVJ9wMbduWAzcys9lo2Q4SkIcBv8uM36s4ZIszMus4ZIszMrGW17CMkIuJJ0qw8\nMzPrY9xzMjOzplP3nlN+eODtEfFM/vwk0BYRy2q8n9tIj8AAODoiftRB3W2Am0hP110LuCQiLq1l\ne3rKKYsax7MjzeqvET2nMcAWtdiQpIrBNSIOjYjlwCDgpE429Szwj/keqL2BM/LDEc3MrAE6DU5V\n5Mw7OOele0DSDTkTA5LOybnqFki6TMloUtaICTlP3np5N/+e159fyHe3gaTxeRsPSjosl4/J+7kF\nuF3S5pKm5u0tkDQi13tS0qbA+cB2efm4cscYEX+PiNKNwetUOi/OEGFmVh/V9Jw6ypk3n5QeaFRE\n7AHMAr6S6/4gIvbMU73XAz4eETfmOsdExLBCGqJlef0fk27iBTgLuDPnuzuA9BTcDfKyfYFjI+JA\n0tDdpNzr+SDpvqaiM4DH8/5OrXSQkrbK90s9DVxQGnYsiojLIqItItoGDx7cyWkzM7PuqiY4dZQz\n7zVgJ2B6vkH2WGCbvN4BSs9Omg8cCOzcwT5uKuxrSH5/MGl4bQ4wBVgX2DovmxwRf8nv7weOkzQW\n2DUiXqnimN4hIp7OCWS3J93Yu1l3tmNmZj3X6YSITnLmLSYFiqOK60hal5Txuy0ins6BY90OdlMa\nUnur0CYBn4mIR9pte2/gr4X2TZX0IVLevWskjYuIq+mmiHhG0kJS8L2xu9upNV+UN7P+pNoJEaWc\neVNJ6YhOJA2f3QvsJ2l7AEnrS9qRVYFoWb4GNbqwrVeAgVXscxLpWpTytncvVynPtHs+Ii4Hfgrs\n0a5Kp/uTtKVWPVNqI1KW8kc6WsfMzHpPtcGpbM68iFhKmn13Xb5ecy8wNM+Su5x0TepXpKG3kiuB\nS9tNiCjnXNK07nlKj2uv9BDAkcAcSQ+Sngt1cXFhRLxAGnZcUGlCBPAPwMycc+8u4DsRMb+DtpmZ\nWS9q2dx6jebcemZmXefcemZm1rJaNrdee9VkKZe0K+980u2KiNi7F5tmZmZd1GeCUzXydaRKT8Kt\nK6cjah2eKWlWf31tWG8NSZdLWijpdknrSZoiqQ1A0qZ5Wnwp08SvJN2Sn5B7sqSv5GwU90rauKFH\nYmbWj/W14LQD8MOI2BlYTpq915FdSBkm9gLOA/4WEbuTbjT+XPvKTl9kZlYffS04LY6IUvqiYraJ\nSv4QEa/kKfEvAbfk8vnl1nX6IjOz+uhrwWlF4X0p28SbrDrO9lkqivVXFj6vpJ9djzMzayb94Qv4\nSWA4cB+rZ6poKF9kNzOrrK/1nMr5DvBFSfcAmza6MWZm1jlniOgmZ4gwM+u6ajNEODh1k6SlwFON\nbkcv2xRY1uhGNAGfh1V8LhKfh6Q752GbiOh0RpmDk1UkaVY1f+H0dT4Pq/hcJD4PSW+eh/5wzcnM\nzFqMg5OZmTUdByfryGWNbkCT8HlYxeci8XlIeu08+JqTmZk1HfeczMys6Tg4mZlZ03Fw6qckHZ4f\nLbKy9EiRwrL/krRI0iOSDimUfySXLZJ0RqH8/ZJmSnpM0vWS1q7nsfSmSsfcV0gaL+l5SQsKZRtL\nmpx/npMlbZTLJen7+VzMk7RHYZ1jc/3HJB3biGPpCUlbSfqDpIfy78V/5PL+eC7WlXSfpLn5XHw9\nl5f9PZe0Tv68KC8fUthW2e+SqkSEX/3wBfwD8AFgCtBWKN8JmAusA7wfeBxYI78eB7YF1s51dsrr\n/AI4Mr+/FPhio4+vRueo4jH3lRfwIWAPYEGh7NvAGfn9GcAF+f2hwG8BAfsAM3P5xsAT+d+N8vuN\nGn1sXTwPmwN75PcDgUfz70J/PBcCBuT3awEz8zGW/T0HTgIuze+PBK7P78t+l1TbDvec+qmIeCgi\nHimz6DDg5xGxIiIWA4tIz7vaC1gUEU9ExN+BnwOHSRJwIHBjXv8q4FO9fwR1UfaYG9ymmoqIqcBf\n2hUfRvo5wuo/z8OAqyO5FxgkaXPgEGByRPwlIl4EJgMf6f3W105EPBsRD+T3rwAPAe+jf56LiIhX\n88e18iuo/HtePEc3Agfl74VK3yVVcXCy9t4HPF34vCSXVSrfBFgeEW+2K+8LKh1zX7dZRDwL6Usb\neE8u7+r/jZaUh6V2J/UY+uW5kLSGpDnA86QA+ziVf8/fPua8/CXS90KPzkV/eGRGvyXp98B7yyw6\nKyJ+XWm1MmVB+T9kooP6fUFfPrbuqHQ++sx5kjQA+CXw5Yh4OXUAylctU9ZnzkVEvAUMkzQIuJl0\nGeAd1fK/vXIuHJz6sIgY1Y3VlgBbFT5vCTyT35crX0Ya0lgz/9VUrN/qOjoXfdlzkjaPiGfzUNXz\nubzS+VgCjGxXPqUO7awpSWuRAtOEiLgpF/fLc1ESEcslTSFdc6r0e146F0skrQlsSBoq7tHvj4f1\nrL2JwJF5Bs77gR1ID2q8H9ghz9hZm3Thc2KkK59/YNWDHI8FKvXKWk3ZY25wm+phIunnCKv/PCcC\nn8sz1fYBXspDXZOAgyVtlGezHZzLWka+RvJT4KGIuLCwqD+ei8G5x4Sk9YBRpGtwlX7Pi+doNHBn\n/l6o9F1SnUbPDPGrMS/g06S/bFYAzwGTCsvOIo0xPwJ8tFB+KGkW0+OkocFS+bb5P90i4AZgnUYf\nXw3PU9lj7isv4DrgWeCN/P/heNL1gjuAx/K/G+e6An6Yz8V8Vp/l+fn8818EHNfo4+rGedifNOQ0\nD5iTX4f203OxG/BgPhcLgHNyednfc2Dd/HlRXr5tYVtlv0uqeTl9kZmZNR0P65mZWdNxcDIzs6bj\n4GRmZk3HwcnMzJqOg5OZmTUdByczM2s6Dk5mZtZ0/j9bSJCrlAhWUgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10b87f2d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaEAAAEKCAYAAAC7c+rvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xt8VdWZ//HPkzsECJcECHcQioIo\nYkS8tKXaKmpbrKOtbcei4wy96K/t/NpftTPT2utM+5pWO05bO7ZatXVqrXYUq9ZBxVpbRcMd5Bbu\ngUACgUASyPX5/XFW8BATckBOds7J9/16bc8+a6+917M5eB722uusbe6OiIhIFDKiDkBERHovJSER\nEYmMkpCIiERGSUhERCKjJCQiIpFREhIRkcgoCYmISGSUhEREJDJKQiIiEpmsqAPo6QoLC33cuHFR\nhyEiklKWLFmy192LuqqnJNSFcePGUVpaGnUYIiIpxcy2JVJP3XEiIhIZJSEREYmMkpCIiERGSUhE\nRCKjJCQiIpFREhIRkcgoCYmISGSUhERE5G1+9PwG/rppb9LbURISEZFjVB48wo+e30jp1v1Jb0tJ\nSEREjvHcm3sAuOLM4UlvS0lIRESO8dzq3Uwoymfi0H5Jb0tJSEREjjpQ38irm/cxZ+pwzCzp7SUt\nCZlZnpm9bmYrzGyNmX0zlD9gZlvMbHlYpodyM7O7zazMzFaa2Yy4Y80zs41hmRdXfq6ZrQr73G3h\nT8zMBpvZwlB/oZkN6qoNERGB59dW0tLqzOmGrjhI7pVQA3CJu58NTAfmmNmssO3/ufv0sCwPZVcA\nk8IyH7gHYgkFuAM4H5gJ3NGWVEKd+XH7zQnltwMvuPsk4IXwvtM2REQk5o+rdzOiII9pIwu6pb2k\nJSGPqQ1vs8Pix9llLvBQ2O81YKCZFQOXAwvdvdrd9wMLiSW0YmCAu7/q7g48BFwdd6wHw/qD7co7\nakNEpNerbWjm5Y1VXH5m93TFQZLvCZlZppktByqJJZLFYdN3Q3fYXWaWG8pGAjvidi8PZccrL++g\nHGCYu1cAhNehXbQhItLrvbS+ksbmVq44s/v+bZ7UJOTuLe4+HRgFzDSzM4GvAqcD5wGDgdtC9Y7S\nrp9E+fEktI+ZzTezUjMrraqq6uKQIiLp4Y+rd1PYL4dzxw7quvIp0i2j49z9APASMMfdK0J3WAPw\nS2L3eSB2VTI6brdRwK4uykd1UA6wp62bLbxWdtFG+3jvdfcSdy8pKury6bQiIinvSFMLi9ZV8oEp\nw8nM6J6uOEju6LgiMxsY1vsA7wfWxSUHI3avZnXYZQHwqTCCbRZQE7rSngMuM7NBYUDCZcBzYdsh\nM5sVjvUp4Mm4Y7WNopvXrryjNkREerWX1ldS19jCldO6Z1Rcm6wkHrsYeNDMMoklu0fd/Q9m9qKZ\nFRHrGlsOfCbUfwa4EigD6oGbANy92sy+DbwR6n3L3avD+meBB4A+wLNhAfge8KiZ3QxsB647Xhsi\nIr3dUysqKOyXwwUThnRru0lLQu6+Ejing/JLOqnvwC2dbLsfuL+D8lLgzA7K9wGXnkgbIiK9VW1D\nMy+s28NHS0aTldm9cxhoxgQRkV7u+Tf3cKSplQ+fPaLb21YSEhHp5Z5asYsRBXnMGNN9o+LaKAmJ\niPRiB+obeXljFR88ewQZ3Tgqro2SkIhIL/bH1btpanE+dFb3d8WBkpCISK/21MpdjC/M58yRAyJp\nX0lIRKSX2lvbwKub9vHBs4q7ba649pSERER6qUXrKml1uHxq9/5ANZ6SkIhIL/XiukqGDchl6oho\nuuJASUhEpFdqbG7lzxv3csnpQyPrigMlIRGRXql0azW1Dc28b/LQrisnkZKQiEgv9MK6SnKyMrho\nYmGkcSgJiYj0QovWVTJrwhDyc5M5j3XXlIRERHqZLXvr2Ly3jksmR/+8NCUhEZFe5sV1sed8XnL6\nsIgjURISEel1Fq2rZOLQfowZ0jfqUJSERER6k9qGZhZv2cclp0c7Kq6NkpCISC+ydNt+mlqc90yK\n/n4QKAmJiPQqq3bWADBtVEHEkcQoCYmI9CIryw8wvjCfgj7ZUYcCKAmJiPQqq8prmDayZ1wFgZKQ\niEivUXWogV01Rzirh3TFgZKQiEivsWrnAQBdCYmISPdbWV6DGZypJCQiIt1tVXkNE4v6RT5fXDwl\nIRGRXsDdWbmzpscMzW6TtCRkZnlm9rqZrTCzNWb2zVA+3swWm9lGM/utmeWE8tzwvixsHxd3rK+G\n8vVmdnlc+ZxQVmZmt8eVn3AbIiLpbM/BBqoONXBWD+qKg+ReCTUAl7j72cB0YI6ZzQK+D9zl7pOA\n/cDNof7NwH53nwjcFephZlOA64GpwBzgp2aWaWaZwE+AK4ApwMdDXU60DRGRdLeyPAxKGDUw4kiO\nlbQk5DG14W12WBy4BHgslD8IXB3W54b3hO2XWuyZs3OBR9y9wd23AGXAzLCUuftmd28EHgHmhn1O\ntA0RkbS2amcNmRnGlOIBUYdyjKTeEwpXLMuBSmAhsAk44O7NoUo5MDKsjwR2AITtNcCQ+PJ2+3RW\nPuQk2hARSWsrymuYNLQffXIyow7lGElNQu7e4u7TgVHErlzO6KhaeO3oisRPYfnx2jiGmc03s1Iz\nK62qqupgFxGR1OHurCo/0KN+pNqmW0bHufsB4CVgFjDQzNrGB44CdoX1cmA0QNheAFTHl7fbp7Py\nvSfRRvt473X3EncvKSrqGTPNioicrPL9h9lf38RZPex+ECR3dFyRmQ0M632A9wNrgUXAtaHaPODJ\nsL4gvCdsf9HdPZRfH0a2jQcmAa8DbwCTwki4HGKDFxaEfU60DRGRtLWyPDZzdk+8EkrmL5aKgQfD\nKLYM4FF3/4OZvQk8YmbfAZYB94X69wG/MrMyYlcn1wO4+xozexR4E2gGbnH3FgAzuxV4DsgE7nf3\nNeFYt51IGyIi6WzZ9v3kZmVw+vCeNSgBwHQhcHwlJSVeWloadRgiIiftb+75KwCPf/bCbmvTzJa4\ne0lX9TRjgohIGmtsbmXVzhrOGd3z7geBkpCISFpbt/sgjc2tnDNmUNShdEhJSEQkjS3bHpsp4Zwx\nuhISEZFutmz7fob2z6W4IC/qUDqkJCQiksaW7zjAOWMG0lNnKFMSEhFJU9V1jWzdV99j7weBkpCI\nSNpavmM/QI8dGQdKQiIiaWvZ9gNkZliPe5BdPCUhEZE0tXzHASYP60/fnJ7zOO/2lIRERNJQa6uz\nfPuBHjs0u42SkIhIGtpUVcuhhuYePSgBlIRERNLSkm1hUIKuhEREpLu9tnkfhf1ymVCYH3Uox6Uk\nJCKSZtyd1zZXM2vC4B77I9U2SkIiImlm2756dh88wqwJQ6IOpUtKQiIiaebVzfsAlIRERKT7vbZ5\nH0X9czmtqGffDwIlIRGRtBK7H7SPWROG9Pj7QaAkJCKSVrbuq2fPwQZmTRgcdSgJURISEUkjr6XQ\n/SBQEhIRSStt94N6+u+D2igJiYikCXfn1U2pcz8IlIRERNLGlr11VB5KnftBoCQkIpI2Uun3QW2U\nhERE0sRL66sYObBPytwPgiQmITMbbWaLzGytma0xsy+E8m+Y2U4zWx6WK+P2+aqZlZnZejO7PK58\nTigrM7Pb48rHm9liM9toZr81s5xQnhvel4Xt47pqQ0QklTU0t/CXsr3MnlyUMveDILlXQs3Al9z9\nDGAWcIuZTQnb7nL36WF5BiBsux6YCswBfmpmmWaWCfwEuAKYAnw87jjfD8eaBOwHbg7lNwP73X0i\ncFeo12kbyfsjEBHpHqVb91Pf2ML7Jg+NOpQTkrQk5O4V7r40rB8C1gIjj7PLXOARd29w9y1AGTAz\nLGXuvtndG4FHgLkWS/WXAI+F/R8Ero471oNh/THg0lC/szZERFLaonWV5GRmcOHE1LkfBN10Tyh0\nh50DLA5Ft5rZSjO738zaHvs3EtgRt1t5KOusfAhwwN2b25Ufc6ywvSbU7+xY7eOdb2alZlZaVVV1\nwucrItLdFq2v5PwJg+mbkxV1KCck6UnIzPoBjwNfdPeDwD3AacB0oAL4YVvVDnb3kyg/mWMdW+B+\nr7uXuHtJUVFRB7uIiPQcO6rr2VRVx+wU64qDJCchM8smloAedvffA7j7HndvcfdW4Oe81R1WDoyO\n230UsOs45XuBgWaW1a78mGOF7QVA9XGOJSKSsl5aXwnA+yan3j+akzk6zoD7gLXufmdceXFctY8A\nq8P6AuD6MLJtPDAJeB14A5gURsLlEBtYsMDdHVgEXBv2nwc8GXeseWH9WuDFUL+zNkREUtai9VWM\nHdKX8Sk0NLtNMjsPLwJuAFaZ2fJQ9k/ERrdNJ9YNthX4NIC7rzGzR4E3iY2su8XdWwDM7FbgOSAT\nuN/d14Tj3QY8YmbfAZYRS3qE11+ZWRmxK6Dru2pDRCQVHWlq4a+b9nL9eWNSamh2G4tdIEhnSkpK\nvLS0NOowREQ69KcNVcy7/3V+edN5PWp4tpktcfeSruppxgQRkRT2/Jt7yMvO4IIUmqonnpKQiEiK\naml1nl29m0tOH0pedmr+7l5JSEQkRb2xtZq9tQ1cOa2468o9lJKQiEiKemZVBblZGT3qXtCJUhIS\nEUlBraEr7n2Th5Kfm1qzJMRTEhIRSUGl2/ZTdaiBK89K3a44UBISEUlJbV1xl5yeul1xoCQkIpJy\nYl1xFbz3XUX0S+GuOFASEhFJOUu372fPwQauSvGuOFASEhFJOX9YWUFOGnTFwQkkITO72MxuCutF\nYQJQERHpRg3NLTyxfCcfmDKM/nnZUYfzjiWUhMzsDmKThX41FGUDv05WUCIi0rHn36zkQH0THy0Z\n3XXlFJDoldBHgA8DdQDuvgvon6ygRESkY79bsoPigjwunlgYdSinRKJJqDE8j8cBzCz1HlohIpLi\ndtcc4eUNVfzNjFFkZqTeYxs6kmgSetTM/ovYk0z/AXie2FNRRUSkmzy+tJxWh2vPHRV1KKdMQgPM\n3f0HZvYB4CAwGfi6uy9MamQiInKUu/O70h3MHD+YcSn4BNXOJJSEQvfbi+6+0MwmA5PNLNvdm5Ib\nnoiIQGyanq376rn1kklRh3JKJdod9zKQa2YjiXXF3QQ8kKygRETkWI+8voP8nEyunDY86lBOqUST\nkLl7PXAN8J/u/hFgSvLCEhGRNlWHGnhqxS6umTGKvjmpPU1PewknITO7APgk8HQoS68/CRGRHurh\nxdtobGnlxovGRR3KKZdoEvoCcDvwe3dfE2ZLeDF5YYmICMRmSPj1a9t43+QiTivqF3U4p1yiVzP1\nQCvwcTP7W8AIvxkSEZHkeWpFBXtrG7n54glRh5IUiSahh4EvA6uJJSMREUkyd+f+V7bwrmH9uGji\nkKjDSYpEk1CVuz+V1EhEROQYr22u5s2Kg3zvmmmYpccMCe0lmoTuMLNfAC8ADW2F7v77pEQlIiLc\n98pmBvXN5upzRkYdStIkOjDhJmA6MAf4UFg+eLwdzGy0mS0ys7VmtsbMvhDKB5vZQjPbGF4HhXIz\ns7vNrMzMVprZjLhjzQv1N5rZvLjyc81sVdjnbgv/VDiZNkREepK1FQd5fm0l8y4cR152ZtThJE2i\nSehsdy9x93nuflNY/q6LfZqBL7n7GcAs4BYzm0JslN0L7j6J2JXV7aH+FcCksMwH7oFYQgHuAM4H\nZhK7KhsU9rkn1G3bb04oP6E2RER6mp8sKqNfbhY3XZjej25LNAm9FhJIwty9wt2XhvVDwFpgJDAX\neDBUexC4OqzPBR7ymNeITZZaDFwOLHT3anffDywE5oRtA9z91TDD90PtjnUibYiI9Bibqmp5elUF\nN1wwloK+qf/guuNJNAldDCw3s/WhG2uVma1MtBEzGwecAywGhrl7BcQSFdD2fNqRwI643cpD2fHK\nyzso5yTaaB/vfDMrNbPSqqqqRE9TROSU+OmiTeRmZXDzxel9FQSJD0yY03WVjplZP+Bx4IvufvA4\nIzw62uAnUX7ccBLZx93vBe4FKCkp0e+hRKTb7Kiu54nlO5l3wTgK++VGHU7SJfooh20nc3AzyyaW\ngB6OG0m3x8yK3b0idIVVhvJyIP55taOAXaF8drvyl0L5qA7qn0wbIiI9wk9f2kSmGfPfk54/Tm0v\n0e64ExZGqt0HrHX3O+M2LQDaRrjNA56MK/9UGME2C6gJXWnPAZeZ2aAwIOEy4Lmw7ZCZzQptfard\nsU6kDRGRyG3bV8fvSnfw0fNGMbwgL+pwukUyJyG9CLgBWGVmy0PZPwHfI/ak1puB7cB1YdszwJVA\nGbFpgm4CcPdqM/s28Eao9y13rw7rnyX2SIk+wLNh4UTbEBHpCe5cuIGsTOPzafbMoOOx2MAy6UxJ\nSYmXlpZGHYaIpLk1u2q46u5X+Nzs0/jKnNOjDucdM7Ml7l7SVb2kdceJiEjifvDcegr6ZPPp954W\ndSjdSklIRCRiizfvY9H6Kj7z3tMo6JPevwtqT0lIRCRC7s73/7iOof1zufHCcVGH0+2UhEREIvT7\npTtZuv0AX75sMn1y0neOuM4oCYmIRKTmcBP/9uxazhkzkGvPHdX1DmkomUO0RUTkOO5auIHqukYe\nuGkmGRnp+bygruhKSEQkAmt21fDQq1v55PljOXNkQdThREZJSESkm7W0Ol97YjUD++bw5csmRx1O\npJSERES62S/+vJml2w/wL1edkfaPauiKkpCISDdaW3GQH/7vBi6fOoyPpPFjuxOlJCQi0k0amlv4\nx98uZ0CfbP71I9M4zqNteg2NjhMR6SY/en4j63Yf4hefKmFIL3hWUCJ0JSQi0g0Wb97Hf/1pEx8r\nGc37pwyLOpweQ0lIRCTJag438X8fXcHowX35+oemRB1Oj6LuOBGRJPv6k6vZffAIj33mAvJz9bUb\nT1dCIiJJ9MSynTy5fBdfuHQS54wZFHU4PY6SkIhIklTUHOZrT6zm3LGD+Nzs3vWcoEQpCYmIJIF7\nbFaEptZW7vzo2WRl6uu2I/pTERFJgqdXVfD82kq+9IHJjB2SH3U4PZaSkIjIKXagvpFvLFjDtJEF\n3HTRuKjD6dE0TENE5BT77tNr2V/fxIN/N1PdcF3Qn46IyCn0ysa9/G5JOf/w7glMHdF7H9GQKCUh\nEZFTpK6hmdt/v5Lxhfl88f2Tog4nJag7TkTkFPnB/66nfP9hHv30BeRlZ0YdTkpI2pWQmd1vZpVm\ntjqu7BtmttPMloflyrhtXzWzMjNbb2aXx5XPCWVlZnZ7XPl4M1tsZhvN7LdmlhPKc8P7srB9XFdt\niIi8U0u27eeBv27lhlljmTl+cNThpIxkdsc9AMzpoPwud58elmcAzGwKcD0wNezzUzPLNLNM4CfA\nFcAU4OOhLsD3w7EmAfuBm0P5zcB+d58I3BXqddrGKT5nEemFGppbuO3xlRQPyOO2K06POpyUkrQk\n5O4vA9UJVp8LPOLuDe6+BSgDZoalzN03u3sj8Agw12IP4bgEeCzs/yBwddyxHgzrjwGXhvqdtSEi\n8o784s9bKKus5bvXTKOf5oY7IVEMTLjVzFaG7rq2iZRGAjvi6pSHss7KhwAH3L25Xfkxxwrba0L9\nzo4lInLSdh04zI9fLGPO1OG8b/LQqMNJOd2dhO4BTgOmAxXAD0N5R48X9JMoP5ljvY2ZzTezUjMr\nraqq6qiKiAgA331mLa3u/PNVZ0QdSkrq1iTk7nvcvcXdW4Gf81Z3WDkwOq7qKGDXccr3AgPNLKtd\n+THHCtsLiHULdnasjuK8191L3L2kqKjoZE5VRHqBv27ay9MrK/jc7ImMHtw36nBSUrcmITMrjnv7\nEaBt5NwC4Powsm08MAl4HXgDmBRGwuUQG1iwwN0dWARcG/afBzwZd6x5Yf1a4MVQv7M2REROWFNL\nK99YsIZRg/rw6fdOiDqclJW0O2hm9htgNlBoZuXAHcBsM5tOrBtsK/BpAHdfY2aPAm8CzcAt7t4S\njnMr8ByQCdzv7mtCE7cBj5jZd4BlwH2h/D7gV2ZWRuwK6Pqu2hAROVE///NmNuyp5b9uOFe/CXoH\nLHaRIJ0pKSnx0tLSqMMQkR5kw55DfPDuV3j/lKH85BMziA3AlXhmtsTdS7qqp2l7REROQFNLK196\ndAX98rL41twzlYDeIQ1oFxE5AT97aROrdtbwk0/MoLBfbtThpDxdCYmIJGhtxUHufnEjHzyrmKvO\nKu56B+mSkpCISAKaW1r5ymMrGZCXzbfmnhl1OGlD3XEiIgm475UtrNpZw48/cQ6D83OiDidt6EpI\nRKQLW/bWcefCDVw2ZRhXTVM33KmkJCQichytrc7tj68kJyuDb1+t0XCnmpKQiMhx/HrxNhZvqeZf\nrjqDYQPyog4n7SgJiYh0Ym3FQb7z9FpmTy7ioyWju95BTpiSkIhIB+obm7n1v5cysE82P7jubHXD\nJYlGx4mIdOAbC9aweW8dD998vn6UmkS6EhIRaeeJZTt5tLScW2ZP5MKJhVGHk9aUhERE4izdvp+v\nPL6SmeMH88X3T4o6nLSnJCQiEuw8cJj5Dy1h+IA8fva355KVqa/IZNM9IRERoLahmZsfeIOG5hYe\nmX++ZkXoJkpCItLrHWlq4bO/XsLGylp+eeN5TBzaP+qQeg1da4pIr9bY3MotDy/lzxv38m/XTOM9\n7yqKOqReRUlIRHqt5pZWPv+bZbywrpLvXH2mfpAaASUhEemVGppb+Pwjy/jjmt187YNT+NtZY6MO\nqVfSPSER6XUOHmli/kOlvLY5NifczRePjzqkXktJSER6lT0HjzDv/tfZVFXLjz42navPGRl1SL2a\nkpCI9BpLtu3nloeXcuhIE/ffeB7vnqRBCFFTEhKRtOfuPPTqNr7z9JsUF/Thd5+5kCkjBkQdlqAk\nJCJprqa+ia89uZoFK3Zx6elDufOj0ynomx11WBIoCYlI2lq0vpLbH1/JvtpGvnzZu/jc7IlkZOiR\nDD1J0oZom9n9ZlZpZqvjygab2UIz2xheB4VyM7O7zazMzFaa2Yy4feaF+hvNbF5c+blmtirsc7eF\nh32cTBsikl6q6xq57bGV3PTLNyjok83/fO4ibr1kkhJQD5TM3wk9AMxpV3Y78IK7TwJeCO8BrgAm\nhWU+cA/EEgpwB3A+MBO4oy2phDrz4/abczJtiEj6aG5p5cG/bmX2vy/i8aXlfOa9p/HU/7mYaaMK\nog5NOpG07jh3f9nMxrUrngvMDusPAi8Bt4Xyh9zdgdfMbKCZFYe6C929GsDMFgJzzOwlYIC7vxrK\nHwKuBp490TbcveJUnreIdD9357k1e7hr4QbW7znERROH8I0PTWXSMM0B19N19z2hYW1f+u5eYWZD\nQ/lIYEdcvfJQdrzy8g7KT6YNJSGRFNXa6jy/dg//8cJG1uw6yITCfO755AzmnDlcj+NOET1lYEJH\nf1v8JMpPpo23VzSbT6zLjjFjxnRxWBHpbnUNzTy+tJxf/mUrW/bWMWZwX3543dnMnT5CzwBKMd2d\nhPa0dYGF7rbKUF4OxM8cOArYFcpntyt/KZSP6qD+ybTxNu5+L3AvQElJSVfJTUS6gbuzoryG35Xu\nYMGKXRw60szZowfyH9dP58ppxWQr+aSk7k5CC4B5wPfC65Nx5bea2SPEBiHUhCTyHPCvcYMRLgO+\n6u7VZnbIzGYBi4FPAf95Mm0k8VxF5BTYW9vAE8t28mjpDjbsqSUvO4MrzizmhgvGMmPMoK4PID1a\n0pKQmf2G2FVMoZmVExvl9j3gUTO7GdgOXBeqPwNcCZQB9cBNACHZfBt4I9T7VtsgBeCzxEbg9SE2\nIOHZUH5CbYhIz9PQ3MJL66v4/dJyXlhbSXOrM330QP71I9P44NnFDMjTj03ThcUGi0lnSkpKvLS0\nNOowRNJeY3Mri7fs45lVFTy9soKDR5op7JfDNTNGcd25ozTSLcWY2RJ3L+mqXk8ZmCAivVDlwSP8\nZdNeXlpfxYvrKjl0pJm+OZlcPnU4c6eP4OKJhRpokOaUhESk21QePMLrW6t5Y0s1r22uZv2eQwAM\nzs/hijOHc9mU4Vw8qZC87MyII5XuoiQkIklRc7iJFTsOsGLHAVbtrGH1zhp21RwBoE92JueOHcTV\n54zk3ZMKmVI8QFPq9FJKQiLyjjW3tLJhTy3Lduxn+fYDLNtxgLLK2qPbxxfmc+64wfzdqALOGzeY\nKSMGaEi1AEpCInICWlqd8v31bKqqpayylvW7a1m/5yAb9tTS2NwKwKC+2UwfPZC5Z49gxthBTBtV\noNFs0iklIRE5yt05eKSZiprD7Nx/mPL9h9lRXc/WffVs3VfH9n31NLa0Hq1f1D+X04f3Z94FY5k6\nooBzxgxkzOC+mjJHEqYkJNKLNDa3sr26nt01R9hVczj2euAwu8JrxYHD1DW2HLNPXnYGYwfnc1pR\nPpeeMZTTCvtx2tB8Tivqx8C+ORGdiaQLJSGRNHS4sYUte+vYsOcQG/YcYmNlLZsqa9lWXU9L67G/\nDSzsl8OIgX04rSifiycWMnJgH4oH5jFqUF9GDerDkPwcXdlI0igJiaSo+sZmdlQfZtu+Orbuq2PL\n3nq27K1l6956dh88crReVoYxrjCfycP7c9VZxUwoymdEQR+KC/owdECuhkNLpJSERHqwllZn6746\n1lUcoqyylm376tgS7s3sq2s8pu7g/BzGDenLhROHMH5I/tHEM25IPjlZGokmPZOSkEgP0NLqbK+u\np6wyNupsY2Us6WzYc4gjTW8NBBhRkMe4wnwumzqMUYP6MnpwX8YM7sv4IfkU9NUINEk9SkIi3ejQ\nkSa27atny966o8Ocyypr2by37ugQZ4DhA/KYNKwfn5g5ljOK+3NG8QAmDu2nrjNJO0pCIqdQfWMz\nuw4cYcf+esqr6ykPw5zLDxymvPrYLjQzGD2oLxOH9uM97ypi4tB+Rxf9rkZ6CyUhkeNoaXUOHWni\nQH0TBw43sb++keraRqrrGtlb10DVodhSebCBiprDHDzSfMz+OVkZjBrYh5GD+nDZ1GGMHZLPuCF9\nGTskn/GF+bqykV5PSShJDtQ3snVfPRkGhmEW+5dvhoV1LLbNwMwwYtvatmdkxJfF6mRYXJ0M3r6P\nGZkZb9VPV+5OS6vT1OI0tbaRXA2LAAAIDUlEQVTS1NxKY0srTc1OY0sLR5pi7480tdDQ3EpDU6zs\nSFNLbGlu5XBjbL2usZn6xhbqG2LrtQ3N1DU0c+hIbKltaO40jpzMDIr651LUP5cxQ/py/oTBFBf0\nobggj1GD+jB6cF+K+uVqTjSR41ASSpJXyvZy638vizSGDIPMDDsmCb6V/GJJ7mgStLeSmsUlvbZ9\nM0JSOzaRvnUcwvdsW1lH3J2jv1Bxjq67O60OjuMO7tDqTmtINi2tTnOr09ziNLe20tRyap6BlZuV\nQd+cTPrmZJGfG3vtn5fF8AF55OdmMSAvm/55WQzok82gvtkM7JtNQZ8chuTnMLhfDv1zs9I62Yt0\nByWhJJk5bjD331gSvlDf+gI++oUbvnSPbgtfvPFfwB6+qFta39q3pfXtdZ2wT2vseO3XW8KB2o5x\nNKajX/ptx3/reG3l8XG0xh0nvrztwYh+9D+E947R7kvajuartxIYHJMYMzLeSoCZmUZWRuxqLzvT\nyMrMIDvDyM7MiK1nGjlZGWRnxpacrAxys2KveVmZ5GbH3vfJziQvLH2yM8nNytAVikgPoCSUJEMH\n5HHJgLyowxAR6dH0CzYREYmMkpCIiERGSUhERCKjJCQiIpFREhIRkcgoCYmISGSUhEREJDJKQiIi\nEhlr+7W7dMzMqoBtUceRoEJgb9RBnALpch6gc+mJ0uU8oGefy1h3L+qqkpJQGjGzUncviTqOdypd\nzgN0Lj1RupwHpMe5qDtOREQioyQkIiKRURJKL/dGHcApki7nATqXnihdzgPS4Fx0T0hERCKjKyER\nEYmMklCKMbM5ZrbezMrM7PYOtuea2W/D9sVmNq77o0xMAudyo5lVmdnysPx9FHF2xczuN7NKM1vd\nyXYzs7vDea40sxndHWOiEjiX2WZWE/eZfL27Y0yEmY02s0VmttbM1pjZFzqokxKfS4LnkhKfS4di\nT9rUkgoLkAlsAiYAOcAKYEq7Op8DfhbWrwd+G3Xc7+BcbgR+HHWsCZzLe4AZwOpOtl8JPEvsobKz\ngMVRx/wOzmU28Ieo40zgPIqBGWG9P7Chg79fKfG5JHguKfG5dLToSii1zATK3H2zuzcCjwBz29WZ\nCzwY1h8DLjWznvgc60TOJSW4+8tA9XGqzAUe8pjXgIFmVtw90Z2YBM4lJbh7hbsvDeuHgLXAyHbV\nUuJzSfBcUpaSUGoZCeyIe1/O2/8yHq3j7s1ADTCkW6I7MYmcC8DfhK6Sx8xsdPeEdsoleq6p4gIz\nW2Fmz5rZ1KiD6Urokj4HWNxuU8p9Lsc5F0ixz6WNklBq6eiKpv3wxkTq9ASJxPkUMM7dzwKe560r\nvFSTKp9JIpYSm47lbOA/gScijue4zKwf8DjwRXc/2H5zB7v02M+li3NJqc8lnpJQaikH4q8GRgG7\nOqtjZllAAT2ze6XLc3H3fe7eEN7+HDi3m2I71RL53FKCux9099qw/gyQbWaFEYfVITPLJval/bC7\n/76DKinzuXR1Lqn0ubSnJJRa3gAmmdl4M8shNvBgQbs6C4B5Yf1a4EUPdy57mC7PpV3//IeJ9YWn\nogXAp8JorFlAjbtXRB3UyTCz4W33GM1sJrHvkH3RRvV2Icb7gLXufmcn1VLic0nkXFLlc+lIVtQB\nSOLcvdnMbgWeIza67H53X2Nm3wJK3X0Bsb+svzKzMmJXQNdHF3HnEjyXz5vZh4FmYudyY2QBH4eZ\n/YbY6KRCMysH7gCyAdz9Z8AzxEZilQH1wE3RRNq1BM7lWuCzZtYMHAau76H/yLkIuAFYZWbLQ9k/\nAWMg5T6XRM4lVT6Xt9GMCSIiEhl1x4mISGSUhEREJDJKQiIiEhklIRERiYySkIiIREZJSCRJzKz2\nHe7/mJlN6KLOS2ZW8k7rtKtfZGZ/TLS+yDuhJCTSA4W5vzLdfXN3t+3uVUCFmV3U3W1L76MkJJJk\n4Rf5/25mq81slZl9LJRnmNlPwzNi/mBmz5jZtWG3TwJPxh3jHjMrDXW/2Uk7tWb2QzNbamYvmFlR\n3ObrzOx1M9tgZu8O9ceZ2Z9D/aVmdmFc/SdCDCJJpSQkknzXANOBs4H3A/8epiS6BhgHTAP+Hrgg\nbp+LgCVx7//Z3UuAs4D3mtlZHbSTDyx19xnAn4jNdtAmy91nAl+MK68EPhDqfwy4O65+KfDuEz9V\nkROjaXtEku9i4Dfu3gLsMbM/AeeF8t+5eyuw28wWxe1TDFTFvf+omc0n9v9sMTAFWNmunVbgt2H9\n10D8RJdt60uIJT6ITcfzYzObDrQA74qrXwmMOMHzFDlhSkIiydfZQwWP97DBw0AegJmNB74MnOfu\n+83sgbZtXYifk6ttNvIW3vr//h+BPcSu0DKAI3H180IMIkml7jiR5HsZ+JiZZYb7NO8BXgdeIfbQ\nvgwzG0Zs4tA2a4GJYX0AUAfUhHpXdNJOBrGJLAE+EY5/PAVARbgSu4HYRLJt3gWsTuDcRN4RXQmJ\nJN//ELvfs4LY1clX3H23mT0OXErsy34Dsadl1oR9niaWlJ539xVmtgxYA2wG/tJJO3XAVDNbEo7z\nsS7i+inwuJldBywK+7d5X4hBJKk0i7ZIhMysn7vXmtkQYldHF4UE1YdYYrgo3EtK5Fi17t7vFMX1\nMjDX3fefiuOJdEZXQiLR+oOZDQRygG+7+24Adz9sZncAI4Ht3RlQ6DK8UwlIuoOuhEREJDIamCAi\nIpFREhIRkcgoCYmISGSUhEREJDJKQiIiEhklIRERicz/B5WQ16cl4/+KAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10ba72090>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('RMSE on Training set :', 754.26562758443276)\n",
      "('RMSE on Test set :', 786.62305857647812)\n",
      "('r2_score on Training set :', 0.84284107642380568)\n",
      "('r2_score on Test set :', 0.85451327131168908)\n"
     ]
    }
   ],
   "source": [
    "# 3* Lasso\n",
    "#1. 生成学习器实例，LassoCV默认参数可自动确定alpha的搜素范围\n",
    "lasso = LassoCV()\n",
    "\n",
    "#2.模型训练\n",
    "lasso.fit(X_train, y_train)\n",
    "alpha = lasso.alpha_\n",
    "print(\"Best alpha :\" , alpha)\n",
    "\n",
    "#3. 模型性能：cv\n",
    "mse_cv = np.mean(lasso.mse_path_, axis = 1)\n",
    "rmse_cv = np.sqrt(mse_cv)\n",
    "print(\"cv of rmse :\", min(rmse_cv))\n",
    "\n",
    "# 4. 特征重要性\n",
    "#Plot important coefficients\n",
    "coefs = pd.Series(lasso.coef_, index = feat_names)\n",
    "print(\"Lasso picked \" + str(sum(coefs != 0)) + \" features and eliminated the other \" +  \\\n",
    "      str(sum(coefs == 0)) + \" features\")\n",
    "imp_coefs = pd.concat([coefs.sort_values().head(10),\n",
    "                     coefs.sort_values().tail(10)])\n",
    "imp_coefs.plot(kind = \"barh\")\n",
    "plt.title(\"Coefficients in the Lasso Model\")\n",
    "plt.show()\n",
    "\n",
    "#5. 显示不同alpha对应的模型性能\n",
    "plt.plot(np.log10(lasso.alphas_), mse_cv) \n",
    "\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show() \n",
    "\n",
    "#训练误差\n",
    "y_train_pred = lasso.predict(X_train)\n",
    "rmse_train = np.sqrt(mean_squared_error(y_train,y_train_pred))\n",
    "print(\"RMSE on Training set :\" ,rmse_train)\n",
    "\n",
    "#测试误差\n",
    "y_test_pred = lasso.predict(X_test)\n",
    "rmse_test = np.sqrt(mean_squared_error(y_test,y_test_pred))\n",
    "print(\"RMSE on Test set :\" ,rmse_test)\n",
    "\n",
    "r2_score_train = r2_score(y_train,y_train_pred)\n",
    "r2_score_test = r2_score(y_test,y_test_pred)\n",
    "print(\"r2_score on Training set :\" , r2_score_train)\n",
    "print(\"r2_score on Test set :\" , r2_score_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "season_1        -991.864539\n",
       "season_2          -0.000000\n",
       "season_3           0.000000\n",
       "season_4         448.139977\n",
       "mnth_1          -152.357646\n",
       "mnth_2           -46.677131\n",
       "mnth_3           272.393334\n",
       "mnth_4            -0.000000\n",
       "mnth_5           364.227409\n",
       "mnth_6             0.000000\n",
       "mnth_7          -395.772042\n",
       "mnth_8            10.784305\n",
       "mnth_9           641.369561\n",
       "mnth_10          352.506349\n",
       "mnth_11         -209.768543\n",
       "mnth_12         -153.539040\n",
       "weathersit_1     388.138775\n",
       "weathersit_2      -0.000000\n",
       "weathersit_3   -1429.882687\n",
       "weekday_0       -273.619964\n",
       "weekday_1       -134.265169\n",
       "weekday_2        -22.758843\n",
       "weekday_3         -0.000000\n",
       "weekday_4         47.064979\n",
       "weekday_5         77.322211\n",
       "weekday_6         82.220008\n",
       "temp            2964.600668\n",
       "atemp            863.906434\n",
       "hum            -1618.968372\n",
       "windspeed      -1382.149139\n",
       "holiday         -301.586913\n",
       "workingday         1.389914\n",
       "yr              1941.529523\n",
       "dtype: float64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coefs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lasso模型增加了L1正则，系数值进行了收缩，同时有些特征的系数为0。\n",
    "在这个例子中，岭回归模型比Lasso模型性能稍好。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 对测试集进行测试，生成提交文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_test_pred = ridge.predict(X_test)\n",
    "\n",
    "#生成提交测试结果\n",
    "df = pd.DataFrame({\"instant\":testID, 'cnt':y_test_pred})\n",
    "df.to_csv('submission.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 147 entries, 196 to 239\n",
      "Data columns (total 2 columns):\n",
      "cnt        147 non-null float64\n",
      "instant    147 non-null int64\n",
      "dtypes: float64(1), int64(1)\n",
      "memory usage: 3.4 KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "_change_revision": 0,
  "_is_fork": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
